Installing Qwen3.8-27B from Scratch: A Guide to Mac Local Deployment and Performance Tuning

Qwen3.8-27B is here.

Ordinary Macs have a chance to run it.

Memory, speed, context—

This guide explains it all at once.

Two pieces of news recently collided.

On August 14, Qwen3.8-27B officially opened its weights. Less than two weeks later, Apple released the new Mac Studio equipped with M5 Max and M5 Ultra, highlighting local AI performance and up to 512GB of unified memory.

After reading these introductions, it's easy to get the illusion: to run Qwen3.8-27B on a Mac, do you also need to buy the latest Mac Studio, or even go straight for the Ultra?

Actually, it's not that exaggerated.

In the past, 27B Dense models were indeed not the preferred choice for local users. The characteristic of Dense models is that for every token generated, the full main parameters must be read and calculated. On 24GB-class devices, even after quantization, it barely fits into memory, and early community tests often showed only single-digit to low double-digit tokens per second.

In contrast, MoE models like 35B-A3B, while having more total parameters, only activate about 3B parameters per generation, potentially making them several times faster. For Agents that need to continuously read code, call tools, and repeatedly modify files, no matter how strong the model capability is, if every round takes a long time, it's hard to become a daily tool. Therefore, many local players previously prioritized MoE.

Now the situation is starting to change.

Quantization formats, Apple Silicon inference frameworks, and new generations of decoding acceleration methods are gradually maturing, giving 27B Dense models their first chance to balance capability and speed. You don't necessarily need the latest Ultra: 24GB and 32GB Macs can start with the 4-bit version, while those with 48GB or more have more flexible options.

The real question is no longer just "can it load," but how to choose the quantization version, control context and memory, and tune the generation speed to be truly usable.

This article will complete a reproducible deployment from scratch: first calculate the memory requirements, then run the basic speed without acceleration, finally perform A/B testing with the same task, and launch the model as a local API that OpenAI and Anthropic clients can call.

If you don't plan to deploy now, I suggest bookmarking it first. When you upgrade to a larger memory Mac later, or prepare to connect local models into code Agents, knowledge bases, and automation workflows, just follow this guide.

Conclusion First: Can Your Mac Run It?

Looking only at unified memory, you can use this table to decide:

This table is not the absolute boundary of "can the model be lit up," but a suggestion for "can it work stably."

Some 24GB Macs can indeed load 4-bit, but successful loading doesn't mean it's suitable for long-term use. macOS, browsers, development tools, model running buffers, context caches, and DFlash 2 draft models all compete for the same unified memory. The model might look fine at startup, but the most common failure occurs when it starts swapping after inputting a long piece of code.

Additionally, this tutorial only applies to Apple Silicon, which includes the M1, M2, M3, M4, and M5 series Macs. Intel Macs do not follow this MLX route.

What exactly is 27B? Correcting a common misconception

The 'B' in the model name stands for Billion.

So 27B means approximately 27 billion parameters, not 270 billion.

You can think of parameters as a large set of numbers retained after training. For every token the model generates, it must read and calculate these numbers to determine what the next token should be. 27B is like a machine with 27 billion knobs: training is responsible for adjusting the knobs to the right positions, and local inference is responsible for loading these knobs into memory and continuously reading them.

Qwen3.8-27B is a Dense model. Dense can be simply understood as: for every token generated, the main parameters participate in the calculation.

This is different from MoE models with A3B or A10B in their names. For example, a 35B-A3B model might store 35 billion parameters in total, but only activates about 3 billion parameters each time. It still needs to prepare storage space for all weights, but the calculation and memory reading per token are much smaller.

Therefore, you cannot assume that two models have similar speed, memory usage, and capability levels just because they both say "around 30B." Total parameters, active parameters, model architecture, and quantization precision must be considered together.

Qwen3.8-27B is not a traditional "full attention on every layer" model. The official model card shows it consists of 64 layers, using a hybrid architecture of Gated DeltaNet and Gated Attention: roughly every 3 layers of linear attention are interspersed with 1 layer of standard attention. It natively supports a 262,144 token context, possesses image and video understanding capabilities, has thinking mode enabled by default, and allows adjusting reasoning depth through reasoning_effort.

These capabilities explain why it is suitable for code, research, long tasks, and Agents; they also explain why you can't just look at "27B" when deploying.

What level is its capability?

If we roughly categorize local models on personal computers:

• 3B–8B: Fast startup, low occupancy, suitable for general Q&A, simple extraction, and lightweight tool calls; complex tasks are prone to going off track.
• 14B–30B: Currently the most practical high-quality range, starting to reliably handle code generation, long text processing, structured analysis, and Agent work.
• 70B and above Dense: Overall stability is often stronger, but memory capacity and bandwidth requirements increase significantly, and personal deployment costs are much higher.

Qwen3.8-27B sits right at the position where "personal devices can realistically deploy, and capability is sufficient to enter production workflows."

In the official model card, it scored 61.7 on SWE-bench Pro and 73.0 on Terminal Bench 2.1; in the same table, Opus 4.6 Max scored 53.4 and 78.2 respectively. This result indicates that on some coding and terminal Agent tasks, Qwen3.8-27B is qualified to be discussed in the same table as closed-source flagships.

But don't rewrite this as "27B completely surpasses closed-source flagships."

Benchmarks are affected by prompts, sampling parameters, tool environments, testing frameworks, and inference budgets. The official model card also disclosed the harnesses used for different tests. A higher score only means it performed better under those specific test conditions, not that it leads in knowledge breadth, open-ended reasoning, long-text stability, visual capabilities, and real workflows.

A more accurate positioning is: it's not a complete replacement for closed-source flagships, but it is a local model that can seriously get work done.

The real deciding factor is the memory calculation

Many people equate "model parameter count" directly with "running memory": 27B, so 27GB is needed.

This calculation is wrong. The parameter count must be multiplied by how many bits each parameter occupies.

Roughly calculating for 27 billion parameters:

• BF16: 2 Bytes per parameter, original weights about 54GB.
• 8-bit: About 1 Byte per parameter, theoretical value about 27GB.
• 4-bit: About 0.5 Byte per parameter, theoretical value about 13.5GB.

Theoretical values only count the main weights. Real model repositories also include quantization scales, configurations, vocabularies, visual components, etc. The MLX community version on Hugging Face is about 16.1GB for 4-bit and 29.5GB for 8-bit. A text BF16 community conversion explicitly states about 54GB.

This is just "how big the file is," not "how much it occupies after startup." The model will consume at least four types of space when running.

1. Context Cache

The model needs to remember what it has already read, otherwise it would have to recalculate everything from scratch for every new token. The standard attention part uses KV Cache, and linear attention layers have their own states.

The longer the context, the larger the cache. Tests from the mlx-dspark project show that for Qwen3.8-27B at 128K context, the cache might add about 11GB; a full 256K context might add about 23GB.

This is why "model supports 262K" doesn't mean a 24GB Mac should open 262K. The capability limit is what the model can handle, not your machine's comfortable default.

2. Running Buffer and Temporary Activations

The stage where the model reads a long prompt is called Prefill. During this stage, a large amount of input must be processed at once, and memory and calculation pressure may suddenly rise. A memory screenshot when you just say "hello" doesn't represent the situation after pasting 20,000 tokens of code.

3. macOS and Other Applications

Apple Silicon's CPU and GPU share unified memory, which is the basis for MLX's efficiency and the reason memory budgets must be conservative. The model, system, Chrome, Cursor, Docker, and other programs all compete for space in the same pool.

4. DFlash 2 Draft Model

DFlash 2 is not a free switch. It requires loading an additional draft model and corresponding cache. The project provides a peak chat length reference: about 18GB for 4-bit target model plus draft, and about 29GB for 8-bit. This still doesn't reserve space for macOS.

Therefore, the complete formula should be:

Actual Memory = Model Weights + Context Cache + Running Buffer + Draft Model + macOS and Other Apps

Understanding this formula is more important than remembering the speed of any blogger's computer.

4-bit, 8-bit, BF16: How to Choose?

Quantization can be understood as recording model parameters more compactly. The lower the bits, the more memory the model saves, and it's usually faster; the cost is a loss of some precision.

For ordinary Mac users, I suggest choosing like this:

24GB / 32GB: Start directly with 4-bit

Model repository:

1mlx-community/Qwen3.8-27B-4bit

The 4-bit file is about 16.1GB. 24GB can try it, but you should actively close large background applications and start with 8K–16K context. 32GB will be more suitable for daily use.

Don't just keep stacking ultra-long context and DFlash 2 because 24GB "can load." Get it running stably first, then add variables one by one.

48GB / 64GB: Consider 8-bit

Model repository:

1mlx-community/Qwen3.8-27B-8bit

The 8-bit file is about 29.5GB. 48GB is a realistic starting point, and 64GB will be more comfortable. If you value speed, context space, and system margin more, 64GB can also continue to use 4-bit; there's no need to force 8-bit just for "higher precision."

BF16: Don't treat "can fit" as "suitable for use"

BF16 text weights are already about 54GB. A 64GB Mac is theoretically close to fitting it, but after adding the system, cache, and buffer, the margin will be very small. For actual long-term use, it's better to consider 96GB and above.

For most people, the difference in experience between 4-bit and 8-bit is much smaller than the difference caused by "starting to swap due to insufficient memory." Once continuous swapping occurs, no amount of quantization precision can save the response speed.

Pre-deployment Preparation: Check Chip, Memory, and Disk

First, open the terminal and confirm machine information:

1system_profiler SPHardwareDataType

You need to see an Apple M-series chip and unified memory capacity.

Then check the disk:

1df -h .

It is recommended to leave at least twice the model volume in available space. The download process may generate cache, followed by draft models, multiple quantization versions, and logs. It's best to prepare more than 35GB of free space for 4-bit and more than 60GB for 8-bit.

This tutorial uses uv to manage the Python environment. If not installed:

1brew install uv

Create an independent directory and virtual environment:

1mkdir -p qwen38-local/models
2cd qwen38-local
3
4uv venv .venv
5source .venv/bin/activate

The benefit of this is not just "looking professional," but avoiding mutual pollution of dependencies between MLX, Transformers, and other projects. If you don't want to use it later, just delete this project directory.

Install the necessary tools:

1uv pip install -U huggingface_hub mlx-dspark

mlx-dspark currently requires Apple Silicon and Python 3.10 or above, and will automatically install mlx-lm, mlx-vlm, and appropriate MLX dependencies.

Downloading the Model: Don't click files one by one in the browser

Large models are usually split into multiple weight shards. Downloading them one by one in a browser is prone to interruption, missing files, and inconvenient resumption. A more reliable method is to use the official Hugging Face hf command.

4-bit Download Command

1MODEL_DIR="$PWD/models/Qwen3.8-27B-4bit"
2
3hf download mlx-community/Qwen3.8-27B-4bit \
4  --local-dir "$MODEL_DIR"

8-bit Download Command

1MODEL_DIR="$PWD/models/Qwen3.8-27B-8bit"
2
3hf download mlx-community/Qwen3.8-27B-8bit \
4  --local-dir "$MODEL_DIR"

The new version of Hugging Face Hub uses Xet chunked downloads, which defaults to adaptive concurrency based on the network. Most people don't need to copy the old hf_transfer configuration from previous tutorials.

You might also see this "high-performance download" switch:

1HF_XET_HIGH_PERFORMANCE=1 hf download ...

Don't enable it blindly. Hugging Face official documentation states it increases concurrency, buffering, and CPU usage, making it more suitable for high-bandwidth machines with at least 64GB of memory. Low-memory Macs might actually be slower due to resource contention. 24GB and 32GB machines should use default settings first.

After downloading, check the directory size:

1du -sh "$MODEL_DIR"

First Run: Test Basic Speed First, Don't Rush to Enable DFlash 2

The most common mistake in deploying local models is turning on ten optimization options at once. In the end, it might run fast, but you don't know who to credit; if it runs slow, you don't know who to turn off.

The correct order is to run a baseline first.

Prepare a fixed prompt, preferably close to your real work. For example, if you mainly use it for coding, you can use:

1Please implement a thread-safe cache in Python that supports expiration time and LRU eviction. Explain the design first, then provide the full code and tests.

Baseline test:

1mlx-dspark generate \
2  --model "$MODEL_DIR" \
3  --mode baseline \
4  --prompt "Please implement a thread-safe cache in Python that supports expiration time and LRU eviction. Explain the design first, then provide the full code and tests." \
5  --max-new-tokens 600

Record four numbers:

1. Model loading time.
2. Prompt processing speed (Prefill tok/s).
3. Time to first token (TTFT).
4. Formal generation speed (generation tok/s).

Generation speed determines "how fast words come out one by one," while Prefill and TTFT determine "how long you have to wait after pressing Enter." For code Agents, every round might require re-reading a large amount of system prompts and code, so Prefill often affects the user experience more than pure generation speed.

During testing, also open "Activity Monitor → Memory" to observe memory pressure and Swap. Yellow doesn't necessarily mean an immediate problem, but if Swap continues to rise, it means this configuration has no stable margin.

Don't just run 50 tokens. Short answers will make loading and warm-up time account for too high a proportion and won't show the true speed during continuous generation. It is recommended to generate at least 400–1000 tokens.

How does DFlash 2 make 27B run faster?

Ordinary decoding is serial. Qwen3.8-27B generates one token, the full target model runs once; generates the next, and runs again. Generating 1000 tokens requires approximately 1000 consecutive rounds.

DFlash 2 adds a lighter draft model. The draft model first proposes a set of candidate tokens in parallel, and then the 27B main model verifies them collectively. Correct guesses can be accepted multiple at a time, while incorrect ones are corrected by the main model.

You can think of it as:

• The draft model is an assistant responsible for quick drafting.
• The 27B main model is the editor-in-chief with final decision-making power.
• The more the assistant guesses correctly in a row, the fewer full rounds the editor-in-chief needs to take.

The draft model does not decide the output independently. The DFlash 2 model card states that under greedy decoding, the output is consistent with the target model; during random sampling, it maintains the distribution of the target model.

It's also not guaranteed to accelerate in every scenario.

If the task makes the draft model easy to predict, such as code completion or long text with stable formatting, the acceptance length is usually higher; if the content jumps significantly, the answers are very short, or sampling randomness is high, the draft is often rejected, and the extra calculation might consume the gains.

Enabling DFlash 2: Let the tool calibrate itself, don't copy others' parameters

First run the project's built-in benchmark:

1mlx-dspark benchmark \
2  --model "$MODEL_DIR" \
3  --modes dflash \
4  --caps auto \
5  --trials 3

Explicitly specify --modes dflash here because the current version of the benchmark defaults to testing DSpark and lookup and won't automatically switch to Qwen3.8-27B's DFlash 2. The first run will download the matching draft model; --caps auto will test appropriate draft caps based on your Mac, target model, and quantization version. M1 Max, M4 Pro, and M5 Max have different memory bandwidths and calculation costs, so the optimal parameters shouldn't be exactly the same.

Therefore, it is not recommended to permanently copy --max-draft 7 just because you saw someone else write it. Let the automatic calibration give the answer first, then retest with real tasks.

Use the same prompt to enable auto mode:

1mlx-dspark generate \
2  --model "$MODEL_DIR" \
3  --mode auto \
4  --prompt "Please implement a thread-safe cache in Python that supports expiration time and LRU eviction. Explain the design first, then provide the full code and tests." \
5  --max-new-tokens 600

Now compare it with the baseline:

• Is the output text consistent?
• Has TTFT significantly lengthened?
• How much has generation tok/s improved?
• What is the mean accept length?
• Has peak memory and Swap worsened?

These commands use greedy decoding by default, so the output text of baseline and auto should be consistent, except for very few floating-point tie cases. If the answers are significantly different, check if the prompt, thinking mode, sampling parameters, and software version are identical before discussing speed. During random sampling, DFlash 2 maintains the target distribution but doesn't guarantee the specific words generated twice will be identical.

In the project benchmarks of the mlx-dspark project on M4 Pro 48GB, 8-bit improved from about 8.4 tok/s to 30.5 tok/s, an average of about 3.63 times; 4-bit improved from about 14.7 tok/s to 33.8 tok/s, an average of about 2.30 times.

These are results under specific versions, machines, hot-start states, and test prompts, not a promise. The project's own split data also shows that acceleration ratios differ for chat, code, and math tasks.

The truly useful criterion is not "others reached 30 tok/s," but whether your high-frequency tasks have become faster.

If you usually have the model modify code, test it with modification tasks in real repositories; if you use it to write articles, generate 1500 tokens continuously; if you want to connect an Agent, run a full tool call. Only if the total time for real tasks decreases is DFlash 2 worth keeping on.

Launching the Model as a Local API

After confirming that both basic and auto modes are stable, you can make the model a resident service. For a 24GB Mac, first limit the context to 8K:

1mlx-dspark serve \
2  --model "$MODEL_DIR" \
3  --mode auto \
4  --context-window 8192

32GB can start with 16K; after stabilization, gradually increase to 32K:

1mlx-dspark serve \
2  --model "$MODEL_DIR" \
3  --mode auto \
4  --context-window 16384

After the service starts, check the status in another terminal:

1curl http://127.0.0.1:8080/health
2curl http://127.0.0.1:8080/v1/models

/health will return the actual mode, context limit, and memory warnings; /v1/models will provide the model ID that the client should fill in.

Do not mix up the addresses for the two types of clients:

1OpenAI Base URL: http://127.0.0.1:8080/v1
2Anthropic Base URL: http://127.0.0.1:8080
3Anthropic Messages route: /v1/messages

It provides both OpenAI and Anthropic compatible interfaces. Chat clients, code tools, and Agents that support custom Base URLs can usually be connected.

Perform a conversation test with curl. The following uses the model ID returned for 4-bit as an example; if you downloaded 8-bit, please replace it with the actual value returned by /v1/models:

1curl http://127.0.0.1:8080/v1/chat/completions \
2  -H "Content-Type: application/json" \
3  -d '{
4    "model": "Qwen3.8-27B-4bit",
5    "messages": [
6      {"role": "user", "content": "Explain what unified memory is in three sentences."}
7    ],
8    "max_tokens": 200
9  }'

When using it only on the local machine, 127.0.0.1 is the safest and easiest choice. Some clients force you to fill in an API Key; you can fill in any placeholder string. When authentication is not enabled, the local service will not verify it.

If you need LAN access, only then consider modifying the listening address and firewall. Do not expose an interface without authentication, TLS, or rate limiting directly to the public internet. Just because the model runs locally doesn't mean the service is naturally secure.

How to set context so memory doesn't explode?

The most reliable method is not guessing, but increasing in steps:

1. 24GB starts from 8K, try 16K after stabilization.
2. 32GB starts from 16K, then try 32K.
3. 48GB / 64GB starts from 32K, try 64K as needed for tasks.
4. Only continue increasing to 128K when truly processing ultra-long documents or large codebases.

For every level you increase, repeat the same test: fixed prompt, fixed maximum output, record TTFT, generation speed, peak memory, and Swap.

"Model supports 262K" is a capability parameter, not a default recommendation. For daily chat, writing, and most coding tasks, 16K–32K can already cover many scenarios.

Larger context doesn't mean smarter; stuffing too much irrelevant content might dilute key information, making the model slower, more expensive, and more prone to going off track.

If the service is used for an Agent, prioritize retaining Prefix Cache. System prompts and tool definitions for code Agents are often very long; reusing prefixes between multiple rounds can significantly reduce repeated Prefill.

How to choose thinking mode? The most easily overlooked variable in testing

Qwen3.8 will think before answering by default. For complex code modifications, mathematical reasoning, research analysis, and multi-round Agent tasks, you can keep the default thinking mode; for general chat, translation, summarization, and format conversion, the thinking process often only increases waiting time and output tokens.

If you want to keep thinking but reduce reasoning depth, use the full command:

1mlx-dspark serve \
2  --model "$MODEL_DIR" \
3  --mode auto \
4  --context-window 16384 \
5  --reasoning-effort low

If the task is very direct, you can turn off thinking:

1mlx-dspark serve \
2  --model "$MODEL_DIR" \
3  --mode auto \
4  --context-window 16384 \
5  --no-thinking

These two parameters set the default behavior of the service. Clients that support related fields can also override them per request, so after connecting tools, confirm if the client has quietly changed back to its own default.

There is no single answer suitable for all tasks. "Low" might look faster per round, but might cause the Agent to retry repeatedly due to insufficient analysis, making the overall task slower. The most reliable method is still calculating the total time for the full task, rather than just comparing the first round of answers.

One rule must be remembered: when doing baseline vs DFlash 2 A/B testing, the thinking mode must be identical. If one has thinking on and the other off, the token count and task path change, and the calculated speed has no comparative meaning. Sampling parameters, prompt, maximum output length, context, and cold/hot start states must also remain consistent.

Shortest Deployment Route: Compressing necessary commands together

What was discussed before is why each step is done. If you already understand the principles and just want to quickly reproduce it, you can execute in the following order. The example chooses 4-bit and 8K context, suitable for a conservative start on a 24GB Mac; the actual time for download and benchmark depends on the network and chip and is not included in the "shortest":

1brew install uv
2
3mkdir -p qwen38-local/models
4cd qwen38-local
5uv venv .venv
6source .venv/bin/activate
7
8uv pip install -U huggingface_hub mlx-dspark
9
10MODEL_DIR="$PWD/models/Qwen3.8-27B-4bit"
11hf download mlx-community/Qwen3.8-27B-4bit \
12  --local-dir "$MODEL_DIR"
13
14mlx-dspark generate \
15  --model "$MODEL_DIR" \
16  --mode baseline \
17  --prompt "Explain unified memory and give three suggestions for running local large models." \
18  --max-new-tokens 400
19
20mlx-dspark benchmark \
21  --model "$MODEL_DIR" \
22  --modes dflash \
23  --caps auto \
24  --trials 3
25
26mlx-dspark serve \
27  --model "$MODEL_DIR" \
28  --mode auto \
29  --context-window 8192

The goal of this set of commands is to "run safely first," not to squeeze the hardware. After running successfully, try 16K and 32K contexts in order based on memory margin, or replace the 4-bit repository with 8-bit. Change only one variable at a time for the test data to be meaningful.

After the service is up, don't rush to connect third-party clients; first access /health and /v1/models. The former confirms no memory warnings and that the expected mode is actually enabled, while the latter confirms the model ID. Then complete a long answer of about 400 tokens and observe the memory pressure and Swap in Activity Monitor. If all four are normal, then fill the Base URL into your daily tools. These few minutes of checking can eliminate most "client cannot connect" and "entire machine becomes laggy after running for a while" problems.

How to restart the next day?

The virtual environment and MODEL_DIR are only effective in the current terminal session. When you reopen the terminal the next day, you don't need to re-download or re-install; just go back to the directory, activate the environment, and re-declare the path:

1cd qwen38-local
2source .venv/bin/activate
3MODEL_DIR="$PWD/models/Qwen3.8-27B-4bit"
4
5mlx-dspark serve \
6  --model "$MODEL_DIR" \
7  --mode auto \
8  --context-window 8192

When upgrading tools, execute within the virtual environment:

1uv pip install -U huggingface_hub mlx-dspark

After upgrading, run a short baseline and /health first to confirm the model can still be loaded before resuming long-term service. Inference tools update quickly, and parameters that worked in old versions are not necessarily always the best, so keeping your own baseline records is valuable.

LAN Access: At least add a lock first

The default 127.0.0.1 can only be accessed by the local machine. If you want another Mac or iPad on the same Wi-Fi to call it, you can listen on all network cards and set an API Key at the same time:

1mlx-dspark serve \
2  --model "$MODEL_DIR" \
3  --mode auto \
4  --context-window 16384 \
5  --host 0.0.0.0 \
6  --api-key "Please replace with a sufficiently long random string"

The client replaces 127.0.0.1 with this Mac's LAN IP and sends Authorization: Bearer your_key in the request. Also check the macOS firewall to only allow trusted networks to access port 8080.

This is still just a LAN solution. To access via the internet, you also need TLS, reverse proxy, access control, and rate limiting; do not map 8080 directly on the router. The easiest way is to return to the home network via a trusted VPN and then access the local service.

Common Problem Troubleshooting

1. Model killed by the system halfway through loading

First confirm you chose the right quantization version. 24GB and 32GB should not mistakenly download 8-bit, and definitely don't touch BF16. Close Docker, virtual machines, a large number of browser tabs, and other local models, then retry 4-bit.

2. Can run, but the whole Mac becomes very laggy

Open Activity Monitor to look at Swap. If Swap continues to rise, shorten the context first, then turn off DFlash 2. Don't just stare at the model process's own numbers, because unified memory pressure is caused by the entire system together.

3. DFlash 2 is actually slower

Confirm comparison conditions are consistent: same prompt, same output length, same thinking mode, same cold start or hot start. Short answers are not suitable for judging speculative decoding gains. Run more than three rounds and test with real long tasks.

If it's still slower, it means the current task acceptance rate is low, or the extra memory brought by the draft model caused the system to start swapping. Turning it off is not a failure; a stable baseline is already an effective solution.

4. First token is very slow, but subsequent generation is okay

This is a Prefill bottleneck. Check if the input is too long, if a large number of irrelevant files are repeatedly stuffed in every round, and if Prefix Cache is hitting. For Agents, optimizing prompt length is often more effective than continuing to pursue generation tok/s.

5. Download speed is very slow or interrupted

Just rerun the same hf download command to utilize cache and resumption. Do not delete the unfinished directory and start from zero. When Hugging Face access is unstable, consider the official recommended ModelScope route.

6. Want it to recognize images

Distinguish between "model has visual capability" and "current service supports visual input." The aforementioned MLX repository retains visual components, but mlx-dspark currently provides a text inference service; image content sent to it will not enter the model.

To test images, you need to temporarily bypass DFlash 2 and use mlx-vlm instead:

1uv run python -m mlx_vlm.generate \
2  --model "$MODEL_DIR" \
3  --max-tokens 200 \
4  --temperature 0 \
5  --prompt "Please describe this image." \
6  --image "/absolute/path/example.jpg"

Visual input increases processing complexity and memory occupancy. If the primary use is code, writing, and Agents, get the text chain stable first, then test visual tasks separately.

A Deployment Sequence Least Likely to Fail

An execution checklist:

1. Confirm it's an Apple Silicon Mac.
2. Give up on 27B for 16GB; choose 4-bit for 24GB/32GB; consider 8-bit for 48GB/64GB.
3. Reserve enough disk space for the model and use uv to create an independent environment.
4. Use hf download to download the full repository; don't click weight files one by one in the browser.
5. Run a fixed prompt with --mode baseline first, recording loading, Prefill, TTFT, generation speed, and memory.
6. Start with 8K, 16K, or 32K context; don't open the full 262K directly.
7. Run mlx-dspark benchmark --modes dflash --caps auto --trials 3 to let the tool calibrate to your machine.
8. Compare baseline and auto with exactly the same real task.
9. Only enable DFlash 2 long-term when speed is significantly improved and memory pressure is stable.
10. Finally, start the local API and connect code tools, knowledge bases, or Agents.

The significance of local deployment is not just saving API fees.

When Qwen3.8-27B becomes a local service on your Mac that can be called at any time, you can keep sensitive code and documents on your own machine, process materials offline, and connect it into automation tasks, personal knowledge bases, and long-running Agent workflows.

My own passing line is simple: common tasks don't swap, answer speed is tolerable, and I will actively open it the next day. Only when these three are met is the deployment truly successful.

If you have already got it running, feel free to leave your "chip model, unified memory, 4/8-bit, context length, baseline and DFlash 2 tok/s" in the comments. If there's enough data, I can continue to organize it into a Mac configuration test table.

If you still find deployment troublesome

I have organized the installation commands, model downloads, speed tests, DFlash 2 acceleration, local API startup, and common problem troubleshooting involved in this article into a deployment checklist that can be followed directly:

1https://github.com/wdwxw/macRunqwen38_27b_install

You can copy and execute them in order yourself, or directly give this GitHub repository to Codex or Claude Code, let it read README.md, check your Mac configuration, and complete the installation according to the checklist. This way you don't have to repeatedly find commands from a long article, and subsequent updates and troubleshooting are more convenient.