Using LM Studio With PostQode

PostQode supports running models locally using LM Studio. LM Studio provides a user-friendly interface for downloading, configuring, and running local language models. It also includes a built-in local inference server that emulates the OpenAI API, making it easy to integrate with PostQode.

Website: https://lmstudio.ai/

Setting Up LM Studio

1. Download and Install LM Studio: Download LM Studio from the LM Studio website.
2. Download a Model: Use the LM Studio interface to search for and download a model in GGUF format. Recommended model families:
   • Qwen 2.5 Coder (e.g. qwen2.5-coder-7b-instruct, qwen2.5-coder-14b-instruct, qwen2.5-coder-32b-instruct)
   • DeepSeek Coder / DeepSeek R1 Distill (e.g. deepseek-r1-distill-qwen-14b, deepseek-coder-6.7b-instruct)
   • Mistral / Devstral (e.g. devstral-24b, mistral-7b-instruct)
   • CodeLlama (e.g. codellama-7b-instruct, codellama-13b-instruct)
3. Start the Local Server:
   • Open LM Studio.
   • Click the "Developer / Local Server" tab.
   • Load your downloaded model.
   • Click "Start Server" (runs on http://localhost:1234/v1 by default).

Configuration in PostQode

1. Open PostQode Settings: Click the gear icon () in the PostQode panel.
2. Select Provider: Choose "LM Studio" from the "API Provider" dropdown.
3. Enter Model ID: Enter the file name of the model you loaded in LM Studio (e.g., codellama-7b.Q4_0.gguf). You can find this in the LM Studio "Local Server" tab.
4. (Optional) Base URL: By default, PostQode will connect to LM Studio at http://localhost:1234. If you've configured LM Studio to use a different address or port, enter the full URL here.

Tips and Notes

• Resource Requirements: Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose.
• Model Selection: LM Studio provides a wide range of models. Experiment to find the one that best suits your needs.
• Local Server: The LM Studio local server must be running for PostQode to connect to it.
• LM Studio Documentation: Refer to the LM Studio documentation for more information.
• Troubleshooting: If you see a "Please check the LM Studio developer logs to debug what went wrong" error, you may need to adjust the context length settings in LM Studio.