Skip to main content

Hugging Face Inference

Hugging Face provides access to tens of thousands of open-source models through their Serverless Inference API as well as dedicated, private Inference Endpoints deployed on enterprise GPU infrastructure.

Website: https://huggingface.co/

Getting an Access Token​

  1. Sign Up / Log In: Go to Hugging Face and create an account or sign in.
  2. Navigate to Access Tokens: Go to huggingface.co/settings/tokens.
  3. Create Token: Click Create new token, select the Read or Inference role, and give it a descriptive name (e.g., "PostQode").
  4. Copy the Token: Copy your token (hf_...) and store it securely.

Supported Models​

PostQode supports any OpenAI-compatible or text-generation model hosted on Hugging Face Serverless Inference or private Inference Endpoints. Recommended models include:

Coding & Agentic Execution​

  • Qwen/Qwen2.5-Coder-32B-Instruct - (Recommended for Coding) Flagship open-weights coding model for unified diffs, refactoring, and multi-language software engineering
  • Qwen/Qwen2.5-Coder-7B-Instruct - Lightweight, fast model for lightweight scripting and quick edits
  • mistralai/Codestral-2501 - Specialized 24B code generation and completion model by Mistral AI
  • bigcode/starcoder2-15b - Multi-language code completion model

Deep Reasoning & Planning​

  • deepseek-ai/DeepSeek-R1 - (Recommended for Planning) Frontier 671B reasoning model with comprehensive chain-of-thought capabilities
  • deepseek-ai/DeepSeek-R1-Distill-Qwen-32B - High-efficiency distilled reasoning model optimized for code and mathematics
  • deepseek-ai/DeepSeek-V3 - 671B Mixture-of-Experts foundation model for agentic execution

General Purpose Foundation Models​

  • meta-llama/Llama-3.3-70B-Instruct - Meta's flagship 70B open model for architectural discussions and documentation
  • meta-llama/Llama-3.1-8B-Instruct - Ultra-fast lightweight model for simple completions
  • microsoft/Phi-4 - Compact 14B model with advanced reasoning capabilities

Deployment Options​

You can connect PostQode to Hugging Face models using two primary approaches:

1. Serverless Inference API​

Ideal for experimentation and smaller models without provisioning infrastructure:

  • Direct access via model repository ID (e.g. Qwen/Qwen2.5-Coder-32B-Instruct).
  • Subject to community rate limits and potential cold-start latency.

2. Dedicated Inference Endpoints​

Recommended for production team workflows and massive models (such as 70B+ or DeepSeek-R1):

  • Deploy models onto dedicated cloud GPUs (NVIDIA L4, A100, or H100) via ui.endpoints.huggingface.co.
  • Paste your unique endpoint URL into PostQode's Base URL setting with 100% dedicated throughput and zero cold starts.

Configuration in PostQode​

  1. Open PostQode Settings ().
  2. Select Hugging Face in the API Provider dropdown.
  3. Enter your Hugging Face Access Token (hf_...) into the API Key field.
  4. Specify your model:
    • For Serverless: Enter the repository Model ID (e.g. Qwen/Qwen2.5-Coder-32B-Instruct).
    • For Dedicated Endpoints: Enter your custom Inference Endpoint URL (e.g. https://your-endpoint.endpoints.huggingface.cloud/v1) and target model identifier.
  5. Click Save Settings to start using PostQode.

Tips and Notes​

  • Gated Models: Models like Meta Llama require accepting the community license agreement on Hugging Face before your access token can invoke them.
  • Warm-Up / Cold Starts: Serverless models that are not already warm in Hugging Face's cache may take 20–40 seconds to load on the initial request. Use Dedicated Endpoints for consistent sub-second latency.
Is this documentation incorrect or incomplete? Report an issue on GitHub