For context, my development flow is always:

1. remote ssh to my workstation

2. start/attach to a tmux session

3. sometimes open neovim to do some editing work.

Occasionally, I need to copy large chunk of context from my terminal/neovim content in remote server, and then search or record it to my note using my laptop client.

I usually had problems with the requirement, either within tmux session or content in the NeoVim. By default the copied content goes to the remote server's clipboard instead of my local client clipboard. In the past, I use tmux plugins / vim plugins (e.g. tmux-yank) to solve the issue, and later found the solution did not work well after some time (Might be due to software update). Or has to be configured differently in different OS. That brings a lot of headache as:

• Some plugins using X11 forwarding require configuring pbcopy/pbpaste, xclip, xsel, wl-clipboard in remote machines. This is not very easy to manage as it requires XWindow, Wayland environment. And you also need ssh -X/-Y to make it available.

• In-compatibility issues might occur if my local client and remote server is not using the same OS. E.g. macos client and ubuntu server. Stange behavior like copy/paste can work in tmux, or NeoVim. But not NeoVim inside tmux.

• Even if fallback to use mouse select terminal content, then use copy/paste shortcut some time conflict with terminal default function. E.g. iTerm has the mouse reporting feature. And if do above, you will see mouse reporting has prevented making a selection warnings.

Recently I just found a more stable and easy way: ANSI OSC52 sequence.

How it works?

You copy text in tmux or Neovim on the remote server. The application sends an OSC 52 escape sequence through the SSH connection. Your local terminal receives this sequence and updates your local system's clipboard.

Requirements:

• A compatible local terminal emulator. Common examples include iTerm2, Kitty, WezTerm, Alacritty, and recent versions of GNOME Terminal.

• tmux (version 2.6+) on the remote server.

Setup:

1. Add the following line to your ~/.tmux.conf file on the remote server:

    set-option -g set-clipboard on
   

2. In NeoVim init.lua

    vim.g.clipboard = 'osc52'
    vim.o.clipboard = 'unnamedplus' # optional if you don't want to overwrite default yank behavior.
   

3. Finally, after you reload tmux, you can use test the copy

    "+y to yank the NeoVim content into your local client clipboard.
    Or y if you set the 'unnamedplus'
   

The second point is especially important, because my setup involves multiple layers (Neovim -> tmux -> SSH -> Your Terminal), and NeoVim seems can't reliably auto-detect that it should use OSC 52. Most tutorial mentioned that NeoVim natively support OSC 52 since this PR, and no setup is needed. But I found in my setup NeoVim content would not be copied to client clipboard, unless the step 2 is done.

Hopefully, this is helpful for people using similar situation as my setup.

What exactly does that mean?

A fundamental knowledge is, for most part, the model is composed of the model's parameters (often called weights and biases). Each parameter is a numerical value that the model learned during training and the precision of these numbers dictates how much space they occupy.

Common data types used in LLM training and inference include:

• FP32 (32-bit floating point): each parameter takes 4 bytes

• FP16 (16-bit floating point) or BF16 (BFloat16): each parameter takes 2 bytes

• INT8 (8-bit integer): each parameter takes 1 byte

• INT4 (4-bit integer): each parameter takes 0.5 bytes

NOTE: Detail explanation of these data types refer to this guide

With above knowledge, it is very easy to estimate the model size using this formula:

$$\text{model_size} = \text{number_of_parameters} \times \text{bytes_per_parameter}$$

How do we calculate the estimated size?

Take an example of Deepseek R1 from Ollama

We know that:

1. There is 671 billion parameters

2. It is using Q4_K_M quantization, that means 0.5 bytes per parameter

Therefore, we can calculate the space usage using

$$671,000,000,000 \text{ parameters} \times 0.5 \text{ bytes/parameter} = 335,500,000,000 \text{ bytes}$$

And the final result is around 312.4GiB, this is closer to the 404 GiB size. The additional size in the actual file can be attributed to the overhead of the Q4_K_M quantization (the scaling factors and other metadata that add to the size) and the model's architecture itself, which includes more than just the parameters.

Overall, using above method should give us a rough estimated scale of the model size, and help us design the system using LLM.