How Can ChatGPT Benefit Pharmacy: A Case Report on Review Writing

2. Materials and methods

A review draft based on the topic of “lipid-based drug delivery systems” was generated using ChatGPT (https://chat.openai.com/, ChatGPT Feb 13 Version). The history of questions (raised by the author) and responses (produced by ChatGPT) were supplied as Supporting Information. The reliability of ChatGPT in writing review article was evaluated by comparing the ChatGPT-generated review to peer-reviewed research and review papers on the same topic.

3. Results and Discussion

In this study, ChatGPT was asked to write the paragraph based on our questions. As shown in the detailed conversation in the Supporting Information, in our first attempt, we tried to instruct ChatGPT to list a concise outline for a scientific mini review article relating on the topic “lipid-based drug delivery systems” and then write a corresponding scientific review article based on the above outline with detailed examples. As instructed, ChatGPT can give reasonable outline for the selected topic and write accordingly based on it. However, due to the length limitation of this ChatGPT version, the given words are restricted below 600 (also proved by the response in the question: “Please provide scientific information on “Applications of lipid-based drug delivery systems” based on the outline you give with detailed examples”), which makes it impossible to write a decent mini review article in a direct way. Therefore, in the following questions, we have to change another way by asking ChatGPT only one question related to one small subtitle in order to give full information.

As we analyzed the review draft given by ChatGPT, we can find two obvious characteristics: 1) the wording is conclusive and too general without specific samples adhere to the topic; 2) lack of citations. The first characteristic might due to the reason that ChatGPT can not fully understand the requirement of the author, which can be improved to some extent with step-by-step conversation until reaching a consensus. However, the second characteristic, or rather called error, is more fatal. As citation is the foundation of a review article, which is critical for readers to judge the accuracy and reliability of the article. It was suggested that due to rights preservation of many publishers, ChatGPT might not be able to freely access the content of many scientific papers, which makes it a shortage in writing scientific articles. In another attempt, we tried to instruct ChatGPT to give specific references based on the given information with the following questions: “Please list the references for the examples of chemotherapeutic agents that have been delivered using lipid nanoparticles for cancer treatment”, we received the following answer: “Certainly! Here are the references for the examples of chemotherapeutic agents that have been delivered using lipid nanoparticles for cancer treatment: 1. Kim, T. Y., Kim, D. W., Chung, J. Y., Shin, S. G., Kim, S. C., & Heo, D. S. (2004). Phase I and pharmacokinetic study of Genexol-PM, a cremophor-free, polymeric micelle-formulated paclitaxel, in patients with advanced malignancies. Clinical cancer research, 10(11), 3708-3716. 2. Gradishar, W. J., Tjulandin, S., Davidson, N., Shaw, H., Desai, N., Bhar, P., ... & O’Shaughnessy, J. (2005). Phase III trial of nanoparticle albumin-bound paclitaxel compared with polyethylated castor oil-based paclitaxel in women with breast cancer. Journal of clinical oncology, 23(31), 7794-7803. 3. LoRusso, P., Krishnamurthi, S. S., Rinehart, J. J., Nabell, L. M., Malburg, L., Chapman, P. B., ... & Hamilton, A. (2010). Phase I clinical evaluation of CRLX101, a cyclodextrin-containing polymer-camptothecin conjugate, in patients with advanced solid tumors. Clinical cancer research, 16(4), 1257-1267. 4. Kim, T. Y., Kim, D. W., Kim, J. H., Kim, H. K., Heo, D. S., & Bang, Y. J. (2007). Phase I and pharmacokinetic study of NANO-paclitaxel, a novel Cremophor-free and albumin-free formulation of paclitaxel, in patients with advanced non-small-cell lung cancer. Annals of Oncology, 18(12), 2003-2009. I hope this helps! Let me know if you have any further question.” (data not shown). After double check the accuracy of the given references using Google Scholar, we can not identify reference 3 and 4, which means ChatGPT reach this answer without truly assessing to the article but merely relayed from other sources. Moreover, we can not rule out the possibilities that ChatGPT might fabricate the references by suturing different pieces of information, which deserves to be further evaluated by future studies. Moreover, the given samples in references 1 and 2 do not math the question, as the carriers for Genexol PM, albumin-bound paclitaxel and polyethylated castor oil-based paclitaxel are all out of the generally recognized scope of lipid carriers. [21]

Next, we performed more detailed evaluation on this review from two aspects: 1) structure of the outline and 2) the accuracy of the contents. In general, the given outline is acceptable as it is somehow reasonable. However, it was also noted that there are still some flaws. For example, in section II (Types of lipid-based drug delivery systems), liposomes, solid lipid nanoparticles (SLNs), nanostructured lipid carriers (NLCs) and lipid-based microemulsions as four most widely adopted kinds are involved, while lipid-drug conjugate, which represents another important specie of lipid-based drug delivery systems, are neglected. [22,23,24]. Moreover, in section IV (Applications of lipid-based drug delivery systems), cancer therapy, treatment of neurological disorders and delivery of vaccines are listed as equal subtitles. However, as far as we are concerned, cancer and neurological disorders are applications from the perspective of diseases while the delivery of vaccines are application from the perspective of drugs, which are not fully equivalent. The best solution is to divide this subtitle into two aspects as diseases and drug, and filled the corresponding applications with detailed examples.

As for the contents, due to the fact that ChatGPT prompt to give conclusive wording under the given conversation, we have to admit that most of the given information, although might be controversial in particular cases, are acceptable and applicable to this topic. However, after carefully checked the whole draft created by ChatGPT, we also found some fundamental mistakes which severely influence the reliability of the contents (Table 1).

Moreover, there are also some improper uses of examples which are commented in Table 2.

Finally, there are many inconsistencies among the draft. The authors have selected some representative examples in Table 3.

4. Conclusion

In conclusion, here in our study, we employed the public available ChatGPT and tried to instruct it to generate a mini review on the topic of “lipid-based drug delivery systems” by conversation. ChatGPT can give a considerable amount of contents on this topic, but is unable to draft a complete scientific review due to the word restrictions. After carefully study the outline and contents of the review draft produced by ChatGPT, we found the following shortages: 1) Conclusive wording which is too general and not specific enough to the topic; 2) Lack of literature citations; 3) Existence of fundamental mistakes and even potential suturing of information for misleading; 4) Improper use of examples; 5) Inconsistence among the draft without strong logic. Based on the above reasons, we can conclude that it is insufficient for ChatGPT to write a rigorous scientific review but more suitable to be a useful tool as electronic encyclopedia, which can also give some benefit for pharmaceutical education and interchange.