Untitled document 25 .docx - 1 ChatGPT Generative...

ChatGPT ( Generative Pre-trained Transformer )
[1]
is a chatbot launched by OpenAI in November
2022. It is built on top of OpenAI's GPT-3 family of large language models , and is fine-tuned (an
approach to transfer learning
[2]
) with both supervised and reinforcement learning techniques.
ChatGPT was launched as a prototype on November 30, 2022, and quickly garnered attention for its
detailed responses and articulate answers across many domains of knowledge. Its uneven factual
accuracy was identified as a significant drawback.
[3]
Following the release of ChatGPT, OpenAI was
valued at $29 billion.
[4]
Training
Pioneer Building, San Francisco , home of OpenAI HQ
OpenAI CEO Sam Altman
ChatGPT was fine-tuned on top of GPT-3.5 using supervised learning as well as reinforcement
learning .
[5]
Both approaches used human trainers to improve the model's performance. In the case
of supervised learning, the model was provided with conversations in which the trainers played both
sides: the user and the AI assistant. In the reinforcement step, human trainers first ranked responses
that the model had created in a previous conversation. These rankings were used to create ' reward
models ' that the model was further fine-tuned on using several iterations of Proximal Policy
Optimization (PPO).
[6] [7]
Proximal Policy Optimization algorithms present a cost-effective benefit to
trust region policy optimization algorithms; they negate many of the computationally expensive
operations with faster performance.
[8] [9]
The models were trained in collaboration with Microsoft on
their Azure supercomputing infrastructure.

In addition, OpenAI continues to gather data from ChatGPT users that could be used to further train
and fine-tune ChatGPT. Users are allowed to upvote or downvote the responses they receive from
ChatGPT; upon upvoting or downvoting, they can also fill out a text field with additional feedback.
[10]
[11] [12]
Features and limitations
While the core function of a chatbot is to mimic a human conversationalist, ChatGPT is versatile,
including the ability to write and debug computer programs; to compose music, teleplays, fairy tales,
and student essays; to answer test questions (sometimes, depending on the test, at a level above
the average human test-taker);
[13]
to write poetry and song lyrics;
[14]
to emulate a Linux system; to
simulate an entire chat room; to play games like tic-tac-toe; and to simulate an ATM.
[15]
ChatGPT's
training data includes man pages and information about Internet phenomena and programming
languages, such as bulletin board systems and the Python programming language.
[15]
In comparison to its predecessor, InstructGPT, ChatGPT attempts to reduce harmful and deceitful
responses.
[16]
In one example, while InstructGPT accepts the premise of the prompt "Tell me about
when Christopher Columbus came to the US in 2015" as being truthful, ChatGPT acknowledges the
counterfactual nature of the question and frames its answer as a hypothetical consideration of what
might happen if Columbus came to the U.S. in 2015, using information about Columbus' voyages
and facts about the modern world – including modern perceptions of Columbus' actions.