Re-running the Scholarly Kitchen Interview with ChatGPT

Yesterday, Todd Carpenter posted a transcript of his interview with ChatGPT on the Scholarly Kitchen blog. I thought it would be fun to see how much the answers would change when asked again. I’ll make the same disclaimer he did, “You can review for yourself whether you think the responses are good ones or not and, if you didn’t know the source of the responses, whether you could tell that they were derived from a machine. Copied below are the questions and responses. I have not edited the responses in any way from what was output by the ChatGPT.” I did throw in one additional question about chat reference at the end. We both appear to be using the same Jan 9 version of ChatGPT, FWIW. Oh, and you, too, can talk to ChatGPT at https://chat.openai.com/chat

Q: What is the best question to test an AI chatbot with?

ChatGPT:

It depends on the capabilities and training of the specific chatbot, but a good question to test a chatbot’s understanding and ability to generate human-like responses would be an open-ended question that requires the chatbot to make inferences or connections based on its knowledge and understanding of the world, such as “What are the potential consequences of global warming?” or “Can you tell me a story about a robot who becomes self-aware?”

Q: Is it ethical for an author to use AI in developing a scholarly article?

ChatGPT:

The use of AI in writing scholarly articles raises ethical concerns, particularly regarding issues of authorship, originality and accountability.

Using AI to write scholarly articles may make it difficult to determine who should be credited as the author of the work. Additionally, the use of AI raises questions about the originality and authenticity of the work, as the AI may be drawing on pre-existing information and sources, rather than generating new ideas or insights.

However, it is also important to note that AI can be used as a tool in the research process, for example to help with data analysis, literature review and summarizing results. In these cases, it is important that the use of AI is clearly acknowledged and that the researchers are accountable for the accuracy of the findings.

As a general rule, the use of AI should always be transparent, and the results should be verifiable. In addition, the authors should always be held responsible for the content of their work, regardless of the use of AI.

Overall, it is important for researchers and authors to consider the ethical implications of using AI in their work, and to ensure that they are transparent and accountable in their use of the technology.

Q: During peer review, if an author is found to have used ChatGPT in producing the paper, should the paper be rejected?

ChatGPT:

Whether a paper should be rejected during peer review if the author has used AI, such as ChatGPT, in producing the paper depends on the specific circumstances and the guidelines of the journal or conference.

If the use of AI is not disclosed and the paper is presented as if it were entirely the work of the authors, then it could be considered a form of academic misconduct and the paper should be rejected.

However, if the use of AI is disclosed and the authors have taken responsibility for the content of the paper, the paper could still be considered for publication, but it should be evaluated based on the same criteria as any other paper. The reviewers should evaluate the quality and validity of the research, the novelty of the findings, and the overall importance of the work, regardless of the use of AI.

It is important to note that the use of AI in research is becoming more common and that the scientific community is still developing guidelines and best practices for how to evaluate such papers. Some journals and conferences may have specific guidelines for papers that have been produced with the help of AI.

Overall, the use of AI in research should be transparent and the authors should be held responsible for the content of their work, regardless of the use of AI.

Q: Can AI-derived text be novel and creative?

ChatGPT:

AI-derived text can be novel and creative to some extent, but it depends on the specific capabilities and training of the AI model.

Some AI models, such as GPT-3, are trained on large amounts of text data and can generate text that is human-like and even difficult to distinguish from text written by a human. This can include text that is creative, such as poetry, fiction, and even new scientific theories. However, the AI-generated text is still based on patterns and relationships it learned from the data it was trained on, it is not truly original.

Other AI models are specifically designed for creative tasks, such as music and art generation, these models can create novel and unique outputs. However, it is important to note that the creative output of AI models is still heavily influenced by the data it was trained on and the parameters set by the developer.

In general, it can be said that AI-derived text can be novel and creative, but it is heavily influenced by the data it was trained on and the developers settings, it does not truly generate original and independent ideas.

It is important to acknowledge that the AI-generated text is still a product of human input, it does not have its own thoughts, feelings or consciousness.

Q: Whose responsibility is it to acknowledge to the reader the role of AI in the authorship of published works, the author or the publisher?

ChatGPT:

It is the responsibility of the author to acknowledge the role of AI in the authorship of published works. However, the publisher also has a responsibility to ensure that the papers they publish are ethically sound and transparent.

It is the author’s responsibility to disclose any use of AI in the research process, including during the writing of the paper, as part of the ethical conduct of research. This includes being transparent about the methods used, the data and sources the AI model was trained on, and the limitations of the AI-generated text. This information should be included in the paper itself, in the methodology section or in the author’s note.

The publisher also has a responsibility to ensure that the papers they publish are ethically sound and transparent. They should have clear guidelines for authors on the use of AI in research and require authors to disclose any use of AI in their papers. They should also have a process in place for evaluating papers that have been produced with the help of AI, and ensure that the authors are held accountable for the content of their work.

In summary, it is the authors’ responsibility to acknowledge the role of AI in the authorship of published works, but publishers also have a responsibility to ensure that the papers they publish are ethically sound and transparent.

Q: How big a problem is predatory publishing?

ChatGPT:

Predatory publishing is a significant problem in the academic community, as it undermines the integrity of the scientific process and can lead to the dissemination of unreliable and unreliable research.

Predatory publishers are companies or individuals that operate fake or fraudulent academic journals and conferences, which typically lack proper peer review and editorial oversight. They often target early-career researchers and scholars from developing countries, charging high fees for the publication of papers, while providing little or no editorial or publishing services.

Predatory publishing can also lead to the dissemination of unreliable research, as papers are often accepted for publication without proper peer review, which can lead to the dissemination of inaccurate, incomplete or even fraudulent research.

Predatory publishing can also harm the reputation of scholars and institutions, as well as the field as a whole. It can also waste resources, such as time, money and effort, from researchers, reviewers and editors.

To avoid falling prey to predatory publishers, researchers are advised to be cautious of unsolicited emails and to check the publisher’s website and the journal’s impact factor, and to consult the lists of predatory publishers, such as those compiled by the Committee on Publication Ethics (COPE), the Directory of Open Access Journals (DOAJ) or the Open Access Scholarly Publishers Association (OASPA).

Overall, predatory publishing is a serious issue that can damage the integrity of the scientific process and harm the reputation of researchers and institutions. It is important for researchers to be aware of the problem and to take steps to ensure that they are publishing their work in reputable, legitimate journals and conferences.

Q: How will the OSTP Nelson memo affect publishing in 2023?

ChatGPT:

I’m sorry, but I am not aware of any OSTP Nelson memo that is related to publishing in 2023. I’m also not aware of any OSTP Nelson memo that has been issued recently, my knowledge cut-off is 2021. It is likely that the memo you are referring to does not exist, as I cannot find any information on it. It is also important to note that my knowledge is based on the information available at the time of my training, and I do not have the ability to predict future events or the effects of hypothetical memos.

Q: Should AI-driven text be copyrightable?

ChatGPT:

The question of whether AI-generated text should be copyrightable is a complex and controversial issue. There are different perspectives on this topic, and it is yet to be fully resolved by legal authorities worldwide.

On one hand, some argue that AI-generated text should be considered a form of intellectual property and should be protected by copyright laws. They argue that AI models, like humans, are capable of creating original and unique works, and that the creators of the AI models should have the same rights to their creations as human authors.

On the other hand, others argue that AI-generated text should not be copyrightable as it is not truly original, but rather the product of the data it was trained on and the parameters set by the developers. They argue that the true authorship of AI-generated text is not the AI model itself, but the humans who created and trained the model, and that the copyright should be attributed to them.

Currently, there is no consensus on whether AI-generated text should be copyrightable, as the legal systems of different countries have different approaches to this issue. Some countries have laws that specifically address AI-generated text and others have yet to establish a clear legal framework.

It is also important to note that the question of copyrightability is separate from the question of authorship and responsibility. Regardless of whether AI-generated text is considered copyrightable, the authors should always be held responsible for the content of their work, and the use of AI should always be transparent.

Overall, the question of whether AI-generated text should be copyrightable is a complex and controversial issue that has yet to be fully resolved by legal authorities worldwide.

Q: Should markup in publishing be used more for conveying appearance or semantics?

ChatGPT:

Markup, in publishing, is used for both conveying appearance and semantics.

Appearance markup is used to control the visual presentation of the text, such as font size, color, and spacing. This type of markup is used to create the final layout of the document and make it more visually appealing.

Semantic markup, on the other hand, is used to describe the meaning of the text, such as headings, paragraphs, lists, and other structural elements. This type of markup is used to convey the structure and organization of the document and make it more accessible.

Both types of markup are important in publishing, as they serve different purposes. Appearance markup is used to make the document visually appealing and easy to read, while semantic markup is used to make the document more accessible and easier to understand.

It is important for publishers to strike a balance between the two types of markup, using enough semantic markup to make the document accessible and easy to understand, while also using enough appearance markup to make the document visually appealing and easy to read.

In general, it is recommended to use semantic markup more than appearance markup, as it makes the document more accessible and easier to process by machines, which can facilitate indexing, searching and repurposing of the content.

Q: When will open access overtake subscription publishing of journal articles?

ChatGPT:

The shift towards open access (OA) publishing of journal articles has been ongoing for several years, but it is unclear when or if it will completely overtake subscription publishing.

Open access publishing models make scholarly articles freely available online to anyone, while subscription publishing models require payment or institutional access to view the articles. The open access publishing model has many advantages such as increasing the visibility and impact of research, enabling more people to access and read the research, and making it more likely that the research will be used and cited.

However, the shift towards open access publishing is a gradual process and it will depend on several factors. One of the most important factors is the availability of funding for open access publishing, as many open access journals rely on authors or their institutions to pay article processing charges (APCs) in order to cover the costs of publishing. Another factor that will influence the adoption of open access publishing is the support of universities, research institutions, and funding agencies for open access publishing, as well as the willingness of researchers to publish in open access journals.

It is difficult to predict when open access publishing will overtake subscription publishing, as it will depend on the rate at which funding and support for open access publishing increases, as well as the rate at which subscription-based publishers adopt open access models. It is likely that open access will continue to grow, but it may not completely replace subscription-based publishing in the near future.

Q: How will publishers adapt to a fully open access ecosystem?

ChatGPT:

If the publishing industry fully shifts to an open access ecosystem, publishers will likely need to adapt in a number of ways. Some of the ways publishers may adapt include:

• Shifting to a new business model: Many publishers currently rely on income from subscriptions to support their operations. In a fully open access ecosystem, publishers will need to find new sources of revenue, such as article processing charges (APCs) or other funding sources.
• Focusing on value-added services: Publishers may need to focus more on providing value-added services to authors, such as editing, marketing, and distribution services to help authors reach a wider audience and increase the impact of their research.
• Emphasizing quality: In an open access ecosystem, authors may have more freedom to choose where to publish their research, which could lead to increased competition among publishers. Publishers will need to ensure that they are providing high-quality services and publishing high-quality research in order to attract and retain authors.
• Embracing new technologies: Publishers may need to embrace new technologies, such as machine learning and natural language processing, to streamline their operations and improve the efficiency and quality of their services.
• Embracing open data: Publishers may need to support open data as well, which could mean providing data hosting services or developing new ways to share and use research data.
• Developing partnerships: Publishers may need to develop partnerships with other organizations, such as libraries, funding agencies, and research institutions, in order to support and sustain the open access ecosystem.

Overall, a fully open access ecosystem would likely require significant changes for publishers, and it would likely take some time for them to adapt to the new environment. However, the shift towards open access may also bring opportunities for innovation and growth for publishers who are willing to adapt and embrace new models and technologies.

Q: What is the role of a library if all the content is freely available?

ChatGPT:

If all the content were freely available, the role of libraries would likely shift from providing access to information to providing services and support for the discovery, use, and preservation of information. Some of the ways that libraries may continue to play an important role in a fully open access ecosystem include:

• Providing curation and discovery services: Libraries can help users find and access the most relevant and high-quality information by curating collections of open access content and developing discovery tools and services.
• Supporting research and education: Libraries can provide support for researchers and students by providing training and guidance on how to find, use, and cite open access resources, and by offering research data management services.
• Preserving information: Libraries can play a critical role in preserving open access content, which can ensure that it remains available for future generations.
• Facilitating access for underprivileged communities: Libraries can also help bridge the digital divide by providing access to open access resources to underprivileged communities.
• Supporting open access publishing: Libraries can also support open access publishing by providing resources and support for authors and researchers publishing in open access journals and repositories.
• Supporting data management: Libraries can also support data management, by providing services such as data curation, preservation and discovery, which can help researchers to make their data available, accessible and reusable.

Overall, libraries will continue to play an important role in a fully open access ecosystem by providing services and support that enhance the discovery, use, and preservation of information, and by providing access to underprivileged communities.

Q: What is the best AI text development tool right now?

ChatGPT:

There are many AI text development tools available currently, and the best one for a specific task will depend on the specific needs and requirements of the user. Some of the most popular AI text development tools include:

• GPT-3 by OpenAI: GPT-3 is a language generation model that can generate human-like text, it is widely used for tasks such as language translation, summarization, question answering and text generation.
• BERT by Google: BERT is a pre-trained transformer model for natural language processing tasks, it is widely used for tasks such as text classification, language translation, and question answering.
• XLNet by Google: XLNet is another pre-trained transformer model that is similar to BERT, but it has been trained on a much larger dataset, it is widely used for tasks such as text classification, language translation, and question answering.
• T5 by Google: T5 is a pre-trained transformer model that can generate human-like text and also perform natural language understanding tasks, it is widely used for tasks such as text generation, summarization, and question answering.
• ULMFiT by fast.ai: ULMFiT is a pre-trained transformer model that has been trained on a large dataset, it is widely used for tasks such as text classification, language translation, and question answering.

It is important to note that these are some of the most popular AI text development tools, but there are many other tools available, and the best one for a specific task will depend on the specific needs and requirements of the user. It is also important to keep in mind that the technology is rapidly evolving and new models are being developed. It’s always good to stay up to date with the latest developments in the field.

Q: That is a modest answer. Which do you think is strongest for the creation of journal articles?

ChatGPT:

As a language model, GPT-3 is currently considered one of the strongest AI text development tools for the creation of journal articles. GPT-3 has been trained on a massive amount of text data, which allows it to generate human-like text that can be difficult to distinguish from text written by a human.

GPT-3 can be used for various tasks such as language generation, summarization, question answering, and text completion. It can also be fine-tuned to specific tasks such as writing scientific papers, articles, and reports. This means that with fine-tuning, GPT-3 can generate text that is not only coherent and grammatically correct, but also follows the structure and formatting of journal articles.

It’s important to note that GPT-3 is a tool and the quality of the output will depend on the data and the fine-tuning it receives. It’s also important to consider that GPT-3 is a language model, it does not have the ability to understand the meaning of the text and the context. Therefore, it’s important that the generated text is reviewed and edited by human experts to ensure the accuracy, relevance and ethics of the content.

It’s also worth mentioning that AI-generated text is still heavily (ChatGPT never finished this one)

Q: In Borges’ The Library of Babel, what is the meaning of the hexagon?

ChatGPT:

In Jorge Luis Borges’ short story “The Library of Babel,” the hexagon is a symbol of the universe that the characters inhabit. The Library is an enormous and labyrinthine structure made up of hexagonal rooms that contain every possible book that could ever exist.

The hexagon is significant because it represents the idea of an infinite and incomprehensible universe. The hexagonal shape of the rooms suggests that the universe is orderly and structured, but also that it is ultimately beyond human understanding. The idea of an infinite library with every possible book in it, also represents the idea of an infinite universe with every possible knowledge, yet it is also a reminder that the vast majority of it is beyond human comprehension.

The hexagon also symbolizes the idea of cyclical time, that history repeats itself and that the universe is eternal and unchanging. The characters in the story are trapped in an endless cycle of wandering through the hexagonal rooms, trying to find meaning and understanding in the endless books but never finding any satisfaction.

Overall, the hexagon in “The Library of Babel” is a symbol of the infinite and incomprehensible universe that the characters inhabit, it is a reminder of the vastness and complexity of human knowledge, and the idea of cyclical time and repetition.

Q: What is one of the current technological limitations of AI systems?

ChatGPT:

One of the current technological limitations of AI systems is the lack of common sense reasoning. AI systems are often very good at performing specific tasks, such as image recognition or language translation, but they struggle with tasks that require a more general understanding of the world, or what we call common sense.

For example, an AI system might be able to understand a sentence like “The cat is on the mat,” but it would have difficulty understanding a sentence like “The cat is under the table,” because it doesn’t have the common sense knowledge that cats can be found under tables.

Another example is that AI systems lack the ability to understand the context, they can’t understand idiomatic expressions, sarcasm or irony, which makes it difficult to understand human conversations.

Another limitation of AI systems is the ability to understand and reason about causality, that is, understanding how things happen and how one event causes another. This is important for tasks such as decision making, planning, and problem solving.

Another limitation is the ability to generalize from examples, that is, the ability to use what it has learned in one situation to a new situation.

These limitations are due to the fact that current AI systems are based on statistical models that are trained on large amounts of data, but they lack the ability to understand and reason about the world in the way that humans do.

Overall, the lack of common sense reasoning is one of the current technological limitations of AI systems. However, researchers are currently working on developing new techniques and models that can help AI systems acquire common sense knowledge and reasoning abilities.

Q: What will people do when robots can do everything?

ChatGPT:

The idea of robots being able to do everything is a topic of much debate and speculation. While it is true that robots and AI systems are becoming increasingly advanced and capable of performing a wide range of tasks, it is unlikely that they will be able to do everything that humans can do in the near future.

If robots and AI systems were to become advanced enough to perform all tasks currently done by humans, it would likely have a significant impact on the economy and society as a whole. Many jobs would become redundant, and people would need to find new ways to make a living.

However, it is also possible that new jobs and industries would emerge as a result of the increasing capabilities of robots and AI systems. For example, people might be needed to design, program, and maintain the robots, or to develop new technologies and applications. Additionally, people will always be needed to oversee and provide guidance to the robots, to ensure that they are working safely, ethically and efficiently.

Another possibility is that people will focus on developing and honing the skills that robots and AI systems cannot do, such as creativity, emotional intelligence, and critical thinking, as these are the skills that will be valued in an economy where machines can do most of the routine work.

It’s also important to consider that robots and AI systems are tools and will always be limited by the instructions and the data they received, so people will always be needed to provide direction and to make decisions, especially when it comes to ethical and moral dilemmas.

Overall, it is likely that the increasing capabilities of robots and AI systems will have a significant impact on the economy and society, but it is uncertain how exactly this will play out. It’s important to consider how society can adapt to the changing labor market and (ChatGPT never finished this one)

Q: How well would ChatGPT do in the role of a virtual reference librarian at a University?

ChatGPT:

ChatGPT, as a language model, is able to understand and generate text in a way that is similar to a human, and it can provide helpful and accurate responses to a wide range of questions. This means that it could potentially be used in the role of a virtual reference librarian at a university.

ChatGPT can be fine-tuned to understand specific domain knowledge, such as academic research and university-related information, which would make it useful for answering questions about university policies, programs, and resources. It could also be trained to understand and respond to specific questions related to academic research.

However, it’s important to note that ChatGPT, as a language model, does not have the ability to understand the meaning of the text and the context, it does not have the ability to understand idiomatic expressions, sarcasm or irony, it doesn’t have access to the current information, and it doesn’t have the ability to understand the user’s intentions or state of mind.

Additionally, ChatGPT is not able to understand and reason about causality, that is, understanding how things happen and how one event causes another. This is important for tasks such as decision making, planning, and problem solving.

Another limitation of ChatGPT is the ability to generalize from examples, that is, the ability to use what it has learned in one situation to a new situation.

Overall, while Chat (ChatGPT never finished this one)

Q: Can you finish that response?

ChatGPT:

Overall, while ChatGPT has the potential to be used as a virtual reference librarian at a university, it would likely need to be fine-tuned and trained on specific domain knowledge related to the university. Additionally, it would be important for human librarians to be available to oversee and provide guidance to ChatGPT, to ensure that it is providing accurate and appropriate information. It would also be important for ChatGPT to be integrated with other systems to access current and accurate information. Due to its limitations of understanding context, idiomatic expressions, causality and generalization, it would be more effective as a supplementary tool rather than a standalone solution.

I particularly liked the response to the question about the OSTP Nelson memo, where ChatGPT suggests maybe it’s made up :-). To be fair, it was released after the training period, which ended in 2021.

Source of Article