Skip to main content

Chat models

info

Head to Integrations for documentation on built-in integrations with chat model providers.

Chat models are a variation on language models. While chat models use language models under the hood, the interface they expose is a bit different. Rather than expose a "text in, text out" API, they expose an interface where "chat messages" are the inputs and outputs.

Chat model APIs are fairly new, so we are still figuring out the correct abstractions.

The following sections of documentation are provided:

  • How-to guides: Walkthroughs of core functionality, like streaming, creating chat prompts, etc.

  • Integrations: How to use different chat model providers (OpenAI, Anthropic, etc).

Get started​

Setup​

Accessing the API requires an API key, which you can get by creating an account and heading here. Once we have a key we'll want to set it as an environment variable by running:

export OPENAI_API_KEY="..."

If you'd prefer not to set an environment variable you can pass the key in directly via the openAIApiKey parameter when initializing the ChatOpenAI class:

import { ChatOpenAI } from "langchain/chat_models/openai";

const chat = new ChatOpenAI({
  openAIApiKey: "YOUR_KEY_HERE",
});

otherwise you can initialize it with an empty object:

import { ChatOpenAI } from "langchain/chat_models/openai";

const chat = new ChatOpenAI({});

Messages​

The chat model interface is based around messages rather than raw text. The types of messages currently supported in LangChain are AIMessage, HumanMessage, SystemMessage, FunctionMessage, and ChatMessage -- ChatMessage takes in an arbitrary role parameter. Most of the time, you'll just be dealing with HumanMessage, AIMessage, and SystemMessage

invoke​

Generic inputs -> generic outputs​

You can generate LLM responses by calling .invoke and passing in whatever inputs you defined in the Runnable.

import { ChatOpenAI } from "langchain/chat_models/openai";
import { PromptTemplate } from "langchain/prompts";

const chat = new ChatOpenAI({});
// Create a prompt to start the conversation.
const prompt =
  PromptTemplate.fromTemplate(`You're a dog, good luck with the conversation.
Question: {question}`);
// Define your runnable by piping the prompt into the chat model.
const runnable = prompt.pipe(chat);
// Call .invoke() and pass in the input defined in the prompt template.
const response = await runnable.invoke({ question: "Who's a good boy??" });
console.log(response);
// AIMessage { content: "Woof woof! Thank you for asking! I believe I'm a good boy! I try my best to be a good dog and make my humans happy. Wagging my tail happily here! How can I make your day better?" }

API Reference:

call​

Messages in -> message out​

You can get chat completions by passing one or more messages to the chat model. The response will be a message.

import { ChatOpenAI } from "langchain/chat_models/openai";
import { HumanMessage } from "langchain/schema";

const chat = new ChatOpenAI({});
// Pass in a list of messages to `call` to start a conversation. In this simple example, we only pass in one message.
const response = await chat.call([
  new HumanMessage(
    "What is a good name for a company that makes colorful socks?"
  ),
]);
console.log(response);
// AIMessage { text: '\n\nRainbow Sox Co.' }

API Reference:

OpenAI's chat model also supports multiple messages as input. See here for more information. Here is an example of sending a system and user message to the chat model:

const response2 = await chat.call([
  new SystemMessage(
    "You are a helpful assistant that translates English to French."
  ),
  new HumanMessage("Translate: I love programming."),
]);
console.log(response2);
// AIMessage { text: "J'aime programmer." }

generate​

Batch calls, richer outputs​

You can go one step further and generate completions for multiple sets of messages using generate. This returns an LLMResult with an additional message parameter.

const response3 = await chat.generate([
  [
    new SystemMessage(
      "You are a helpful assistant that translates English to French."
    ),
    new HumanMessage(
      "Translate this sentence from English to French. I love programming."
    ),
  ],
  [
    new SystemMessage(
      "You are a helpful assistant that translates English to French."
    ),
    new HumanMessage(
      "Translate this sentence from English to French. I love artificial intelligence."
    ),
  ],
]);
console.log(response3);
/*
  {
    generations: [
      [
        {
          text: "J'aime programmer.",
          message: AIMessage { text: "J'aime programmer." },
        }
      ],
      [
        {
          text: "J'aime l'intelligence artificielle.",
          message: AIMessage { text: "J'aime l'intelligence artificielle." }
        }
      ]
    ]
  }
*/

You can recover things like token usage from this LLMResult:

console.log(response3.llmOutput);
/*
  {
    tokenUsage: { completionTokens: 20, promptTokens: 69, totalTokens: 89 }
  }
*/