> ## Documentation Index > Fetch the complete documentation index at: https://togetherai-migration.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Inference > Legacy /inference endpoint. We recommend using the newer completions or chat completions endpoints. ## OpenAPI ````yaml deprecated-spec.json post /inference openapi: 3.1.0 info: title: Together APIs version: '0.2' servers: - url: https://api.together.xyz security: - sec0: [] paths: /inference: post: summary: Inference description: >- Legacy /inference endpoint. We recommend using the newer completions or chat completions endpoints. operationId: inference requestBody: content: application/json: schema: type: object required: - model - prompt - max_tokens properties: model: type: string description: The name of the model to query. default: mistralai/Mixtral-8x7B-Instruct-v0.1 prompt: type: string description: A string providing context for the model to complete. default: [INST] What is the capital of France? [/INST] max_tokens: type: integer description: "The\_maximum number of tokens to generate." default: 512 format: int32 stop: type: array description: >- A list of string sequences that will truncate (stop) inference text output. For example, "" will stop generation as soon as the model generates the given token. default: - - '[/INST]' items: type: string temperature: type: number description: >- A decimal number that determines the degree of randomness in the response. Lower values (e.g. 0.2) result in more consistent outputs, while higher values (e.g. 1.0) generate more diverse and creative results. We generally recommend altering this or `top_p` but not both. default: 0.7 format: float top_p: type: number description: "The\_`top_p`\_(nucleus) parameter is used to dynamically adjust the number of choices for each predicted token based on the cumulative probabilities. It specifies a probability threshold, below which all less likely tokens are filtered out. This technique helps to maintain diversity and generate more fluent and natural-sounding text." default: 0.7 format: float top_k: type: integer description: "The\_`top_k`\_parameter is used to limit the number of choices for the next predicted word or token. It specifies the maximum number of tokens to consider at each step, based on their probability of occurrence. This technique helps to speed up the generation process and can improve the quality of the generated text by focusing on the most likely options." default: 50 format: int32 repetition_penalty: type: number description: >- A number that controls the diversity of generated text by reducing the likelihood of repeated sequences. Higher values decrease repetition. default: 1 format: float stream_tokens: type: boolean description: >- If true, stream tokens as Server-Sent Events as the model generates them instead of waiting for the full model response. If false, return a single JSON object containing the results. safety_model: type: string description: >- Run an LLM-based input-output safeguard model on top of any model. 'n': type: integer description: How many completions to generate for each prompt default: 1 format: int32 examples: Text Generation: value: model: togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1 prompt: The capital of France is max_tokens: 1 temperature: 0.7 top_p: 0.7 top_k: 50 repetition_penalty: 1 responses: '200': description: '200' content: application/json: examples: Result: value: |- { "status": "finished", "prompt": [ "The capital of France is " ], "model": "togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1", "model_owner": "", "tags": {}, "num_returns": 1, "args": { "model": "togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1", "prompt": "The capital of France is ", "temperature": 0.8, "top_p": 0.7, "top_k": 50, "max_tokens": 1 }, "subjobs": [], "output": { "choices": [ { "finish_reason": "length", "index": 0, "text": " Paris" } ], "raw_compute_time": 0.06382315792143345, "result_type": "language-model-inference" } } schema: type: object properties: status: type: string example: finished prompt: type: array items: type: string example: 'The capital of France is ' model: type: string example: togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1 model_owner: type: string example: '' tags: type: object properties: {} num_returns: type: integer example: 1 default: 0 args: type: object properties: model: type: string example: togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1 prompt: type: string example: 'The capital of France is ' temperature: type: number example: 0.8 default: 0 top_p: type: number example: 0.7 default: 0 top_k: type: integer example: 50 default: 0 max_tokens: type: integer example: 1 default: 0 subjobs: type: array output: type: object properties: choices: type: array items: type: object properties: finish_reason: type: string example: length index: type: integer example: 0 default: 0 text: type: string example: ' Paris' raw_compute_time: type: number example: 0.06382315792143345 default: 0 result_type: type: string example: language-model-inference '400': description: '400' content: text/plain: examples: Result: value: Input required '401': description: '401' content: text/plain: examples: Result: value: Missing API key '404': description: '404' content: text/plain: examples: Not Found: value: Invalid Endpoint URL or model name '429': description: '429' content: text/plain: examples: Result: value: Rate limit reached (1 QPS for free, 100 QPS for paid) '500': description: '500' content: text/plain: examples: Result: value: >- Ensure your request is a valid JSON, your API key is correct and you are using the right prompt format '503': description: '503' content: text/plain: examples: Result: value: Engine overloaded, contact support@together.ai '504': description: '504' content: text/plain: examples: Result: value: Timeout, contact support@together.ai '524': description: '524' content: text/plain: examples: Result: value: Cloudflare timeout, contact support@together.ai '529': description: '529' content: text/plain: examples: Result: value: Error on our servers, contact support@together.ai deprecated: false components: securitySchemes: sec0: type: apiKey in: header name: Authorization x-bearer-format: bearer x-default: default ````