Most of the AI tools I use every day are huge models running somewhere in the cloud, so this time I wanted to do something much smaller and more understandable.

My goal was simple: can I take a language model with less than 1 billion parameters, train it on my own laptop, give it my own data, and actually see its behavior change?

I was not trying to build the next ChatGPT. I wanted to understand the whole pipeline myself: dataset, tokenizer, training, LoRA, loss, validation, inference, and finally the fun part, asking the same question before and after training.

The best test ended up being very personal. Instead of asking generic questions like "What is CAN bus?", I asked:

Who is Hendriawan Putra?

The original model had absolutely no idea who I am. It still answered with full confidence. That made this experiment much more interesting.

The model I used

I chose HuggingFaceTB/SmolLM2-360M-Instruct.

I also downloaded the SmolLM2 paper because I wanted to know what this model actually was before I started modifying it. One thing I initially misunderstood is that my LoRA training is not training SmolLM2 from zero. Hugging Face already did the expensive part.

According to the paper, the 360M version was pretrained on about 4 trillion tokens. The smaller SmolLM2 models use the same general Llama-style decoder architecture as the larger SmolLM2 family, with Grouped Query Attention (GQA). For the 360M and 135M models, Hugging Face found that a single-stage training mix with consistently high-quality web, code, math, and synthetic data worked better than copying the exact multi-stage recipe used for the 1.7B model. After pretraining they also did supervised instruction tuning with a filtered SmolTalk dataset, followed by preference training with DPO.

So the checkpoint I start with is already a real language model that knows how to complete text and follow instructions. My experiment starts after all of that.

The whole history of the model I am running is closer to this:

flowchart TB
    A[Hugging Face pretraining<br/>SmolLM2-360M<br/>about 4T tokens] --> B[Hugging Face SFT<br/>filtered SmolTalk]
    B --> C[Hugging Face preference training<br/>DPO / UltraFeedback]
    C --> D[SmolLM2-360M-Instruct<br/>the checkpoint I download]
    D --> E[My LoRA SFT<br/>personal JSONL examples]
    E --> F[My LoRA adapter<br/>8.68M trainable parameters]
    D --> G[Original frozen base weights]
    F --> H[Customized local model]
    G --> H

So when I say "I trained the model," what I really mean in this article is: I trained a LoRA adapter on top of an already-trained SmolLM2 checkpoint. I did not reproduce Hugging Face's 4-trillion-token pretraining run on my laptop. That would be slightly more ambitious than a weekend project. lol.

The important word here is 360M. The model has roughly 360 million parameters. That sounds huge, but for modern LLMs it is actually tiny. Models people normally talk about today can have 7 billion, 70 billion, or far more parameters.

The high-level training story above comes from the SmolLM2 paper. The exact 360M checkpoint dimensions in this table - 32 layers, hidden size 960, 15 query heads, 5 key-value heads, and MLP size 2,560 - come from the actual Hugging Face model configuration loaded by this project.

I deliberately did not train 360 million parameters from zero. That would be a completely different project and would need a lot more data and compute. The original SmolLM2 paper is a good reality check here: even the tiny 360M model was pretrained on around 4T tokens. My 50 examples are not replacing that training. They are only nudging an already-trained model toward my narrow data.

Instead, I started with a pretrained model and added LoRA.

What LoRA means in normal language

Imagine the original model already has a giant brain full of weights. Full fine-tuning would change basically the whole brain. LoRA keeps the original weights frozen and adds a much smaller set of trainable matrices around selected layers.

So my training looked more like this:

flowchart TB
    A[SmolLM2-360M-Instruct<br/>already pretrained + instruction tuned] --> B[Freeze original model weights]
    B --> C[Attach LoRA matrices]
    D[My JSONL examples] --> C
    C --> E[Train only LoRA parameters]
    E --> F[Base model + learned LoRA adapter]
    F --> G[My customized local model]

In my run, exactly 8,683,520 parameters were trainable. That number comes directly from the LoRA configuration and the model dimensions - it is not a random estimate. That is one reason this was easy to run on my RTX 4060.

Where did the 8.68 million trainable parameters come from?

My LoRA settings were:

• rank r = 16
• alpha = 32
• 32 Transformer layers
• LoRA attached to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj

The model config says the hidden size is 960, the MLP intermediate size is 2,560, and the model uses 15 attention heads with 5 key-value heads. Each head is 64 dimensions, so the key/value projection output is 5 x 64 = 320.

For a normal linear layer whose big weight matrix is shaped roughly like out x in, LoRA does not train that whole matrix. It adds two thin matrices, usually called A and B:

A: r x in and B: out x r

So the extra parameter count is:

r x in + out x r = r x (in + out)

For one Transformer layer in this model:

Then the final count is:

271,360 x 32 layers = 8,683,520 trainable parameters

The full model plus adapter contains 370,504,640 parameters in my run, but only those 8.68M LoRA parameters receive gradient updates. That is about 2.34% of the loaded model.

What is the LoRA model actually training?

This part clicked for me once I stopped thinking of LoRA as "training a smaller copy of the model." It is not.

The original projection matrix W stays frozen. LoRA learns a small update ΔW from those two low-rank matrices A and B. During inference, the layer behaves roughly like the original W plus the learned LoRA correction.

flowchart LR
    X[Input activations x] --> W[Frozen original matrix W]
    X --> A[Trainable LoRA A<br/>rank 16]
    A --> B[Trainable LoRA B<br/>rank 16]
    B --> S[Scale by alpha / rank<br/>32 / 16 = 2]
    W --> ADD[Add original + LoRA update]
    S --> ADD
    ADD --> Y[Layer output]

During backpropagation, gradients update A and B, not the big original W. In my setup this happens inside the attention projections (q/k/v/o) and the MLP projections (gate/up/down) in every Transformer layer.

So my personal training is not teaching English from scratch, and it is not rebuilding attention from scratch. SmolLM2 already learned language during pretraining and instruction-following during its own post-training. My LoRA adapter is learning a much smaller correction: associations like "Hendriawan Putra" -> "Indonesia" -> "TMMIN" -> "Design Engineering No. 2", plus the response style in my examples.

What actually happens to one training sentence?

I also wanted the project to make sense instead of hiding everything behind a library call. The flow is basically:

flowchart LR
    A[Text] --> B[Tokenizer]
    B --> C[Token IDs]
    C --> D[Embeddings]
    D --> E[Transformer layers]
    E --> F[Logits]
    F --> G[Next-token probabilities]
    G --> H[Generated token]
    H -->|append token and repeat| E

For example, my dataset contains JSONL rows similar to this:

{"instruction":"Who is Hendriawan Putra?","input":"","output":"Hendriawan Putra is an Indonesian automotive software engineer from Bali. He works for PT Toyota Motor Manufacturing Indonesia (TMMIN)."}

The tokenizer does not understand the sentence like a human. It breaks the text into token IDs. Those IDs become vectors called embeddings. The Transformer processes those vectors and produces a score for every possible next token. Those raw scores are called logits.

During training, the model predicts the next token, compares it with the correct answer, calculates the error using cross-entropy loss, and backpropagation changes the LoRA weights a tiny amount. Repeat that enough times and the behavior starts changing.

That is basically the entire magic loop.

My first dataset: 50 general examples

I first created 50 small examples covering things like Python, C/C++, CAN, AUTOSAR, electronics, mathematics, writing, and basic reasoning.

That was useful to prove the pipeline worked, but it was not a very exciting before/after test. A pretrained language model already knows things like:

• what CAN bus is,
• what an ECU is,
• Python list vs tuple,
• Ohm's law,
• simple arithmetic.

So even if training changed the model a little, the difference was not dramatic.

Then I realized the obvious better experiment: train it on information the base model definitely does not know. Me.

Then I added personal data

I created another 50 examples about my professional background. Things such as:

• I am from Bali, Indonesia.
• I work for TMMIN.
• My 2025 work was in Design Engineering No. 2 and Remote Engine Start.
• In Japan I am doing an ICT training assignment at Daihatsu Motor Company.
• My Japan work is related to ECU Development for a Daihatsu project.
• My technical background includes CAN, UDS, AUTOSAR, embedded software, AI visual inspection, Qt/QML, and so on.

At first I simply mixed those 50 personal examples with the original 50 examples. Total: 100.

I expected the answer to immediately become perfect.

It did not. lol.

Before training: the model just invented a person

Here is a real screenshot from the untouched base model. I ran it locally in Windows Terminal and asked the simplest question possible.

The model answered:

Hendriawan Putra is a Malaysian politician who served as the Minister of National Development and Minister of Education from 2011 to 2015. He was born on 28 May 1962 in Kuala Lumpur, Malaysia.

Literally everything there is invented.

This is a good reminder that a language model does not automatically say "I don't know." If a name looks plausible, a small model can confidently continue with something that also looks plausible.

That is exactly what I wanted for the experiment because now I had a very obvious baseline.

My first personal fine-tune was also bad

I trained the mixed 100-example dataset using LoRA. Technically, the training worked. The loss went down, checkpoints were saved, the adapter loaded, and inference ran normally.

But when I asked the same personal questions, the model still said nonsense such as:

Who is Hendriawan Putra?
→ Malaysian politician...

What is Hendriawan Putra working on in Japan?
→ research institution...

What was Hendriawan Putra doing at TMMIN in 2025?
→ teacher / student / unrelated job...

This was probably the most useful part of the whole project. Adding facts to a dataset does not mean the model stores them like rows in a SQL database.

The problem was my data design.

I had many personal facts, but almost every fact appeared only once. The model saw the word "Hendriawan" a lot, but it did not get enough repeated evidence connecting each fact to the right year, company, and role.

What finally worked: repeated, small, atomic facts

I changed the dataset strategy. Instead of putting one big biography paragraph everywhere, I made the facts smaller and repeated them with different questions.

For example:

Who is Hendriawan Putra?
Tell me about Hendriawan Putra.
Where is Hendriawan Putra from?
Is Hendriawan Putra Malaysian?
Which company employs Hendriawan Putra?

All of those reinforce the same small identity facts.

I did the same for the timeline:

flowchart LR
    A[2025] --> B[TMMIN]
    B --> C[Design Engineering No. 2]
    C --> D[Remote Engine Start]

    E[2026] --> F[DMC Japan]
    F --> G[ICT trainee]
    G --> H[EPS / Software testing]

For the final personal-core experiment I used 50 atomic personal examples, with 45 for training and 5 for validation.

That changed everything.

The training result

This run used five epochs. An epoch simply means one full pass through the training set.

Here is what happened:

The whole personal-core run took about 90.84 seconds on my laptop. The final LoRA adapter had 8,683,520 trainable parameters and the observed PyTorch GPU allocation stayed around 0.91 GB.

The numbers are nice, but I care more about the actual output. A loss number can look great while a model still says something dumb. So I asked the questions again.

After training: now it actually knows some of my data

Here is the real terminal output from the personal-core adapter:

For the exact same identity question, it now says:

Hendriawan Putra is an Indonesian automotive software engineer from Bali. He works for PT Toyota Motor Manufacturing Indonesia (TMMIN).

That is a huge difference from "Malaysian politician."

I also asked:

What was Hendriawan Putra doing at TMMIN in 2025?

The trained model answered:

In 2025 at TMMIN, Hendriawan Putra worked in Design Engineering No. 2 on Remote Engine Start development for IM-V, B-SUV, and B-MPV vehicle projects.

That is the behavior I wanted to see. Same base model. Same laptop. The important difference is the LoRA adapter trained on my data.

Side-by-side result

This is the easiest way to see what changed:

So this is not a perfect "before = stupid, after = perfect" story. Two of the personal relationships became very clear, while one still has a timeline error. That is a much more realistic result.

But it still makes mistakes

This is the part I do not want to hide.

The trained model knows that my Japan assignment is connected to Daihatsu Motor Company, ***, EPS and software testing. But it sometimes binds the wrong year to those facts.

Here is the actual output:

It answered:

In 2025, Hendriawan Putra is an ICT trainee at Daihatsu Motor Company (DMC) in the ****, working on Electric Power Steering (EPS) software testing for ******.

The company, place, system, and project are right. The year is wrong. It should be 2026.

I actually like this failure because it shows what fine-tuning really is. The model did not create a neat table internally with perfectly separated year slots. What I wanted it to keep separate was basically:

flowchart LR
    Y25[2025] --> T[TMMIN]
    Y26[2026] --> D[DMC Japan]

It learned statistical associations between tokens. A tiny 360M model can learn the relationships surprisingly well, but nearby facts can still bleed into each other.

What I learned from the dataset experiments

The biggest lesson was not about GPU speed or model size. It was about the data.

My experiments roughly went like this:

flowchart TB
    A[50 generic examples] --> B[Pipeline works<br/>but before/after difference is boring]
    B --> C[50 generic + 50 broad personal facts]
    C --> D[Still hallucinates<br/>and mixes facts]
    D --> E[50 focused personal examples<br/>with paraphrases]
    E --> F[Much better<br/>but some facts still collide]
    F --> G[50 short atomic personal examples]
    G --> H[Best result]

For a small model, I found that five clean variations of one fact can be more useful than five completely unrelated facts.

Consistency also matters. If one answer says "software engineer", another says "ICT trainee", another says "Toyota engineer", and another includes three different years in one paragraph, a very small model can start combining them in weird ways.

Short and clear examples were much easier for it to learn.

Fine-tuning is not a database

This experiment also changed how I think about using LLMs inside a company.

If I have a fact that changes every day such as production numbers, current project status, latest test results, employee directory, defect counts, I probably should not fine-tune that fact into the model.

For changing factual data, a better design is usually:

flowchart LR
    U[User question] --> R[RAG / database / document search]
    R --> C[Current company context]
    U --> L[Local small LLM]
    C --> L
    L --> A[Company-specific answer]

The model handles language. The database or document retrieval provides the current truth.

Fine-tuning is more interesting for teaching the model how the company talks and how a task should be done.

Where a small local model could actually be useful in a company

This is where I think tiny models are more interesting than people give them credit for. A 360M model is obviously not going to replace a frontier cloud model for difficult reasoning. But it can be extremely cheap and private.

For example, inside an automotive company I could imagine small local models being useful for:

• classifying test or quality reports into known categories,
• turning messy engineer notes into a standard report format,
• summarizing CAN / ECU test logs,
• extracting DTCs, part numbers, symptoms, and test conditions from text,
• rewriting technical explanations for different audiences,
• answering internal terminology questions when combined with RAG,
• creating first drafts of test descriptions or review comments,
• searching engineering documents using natural language,
• running offline on a factory or engineering PC where sending data to a public cloud is undesirable.

The nice part is the deployment cost. A model this small can run on normal hardware. In my experiment the GPU memory used by PyTorch was below 1 GB. Inference can also be quantized further if needed.

For a very narrow task, I would rather test a small local model first than automatically throw a 70B model at the problem.

Of course there is a limit. I would not trust a 360M model to make safety-critical engineering decisions by itself. It can summarize, classify, retrieve, draft, and assist. The final technical decision still needs proper requirements, testing, and human review.

Why local is interesting

There are also practical reasons I like this direction:

Privacy. The prompt can stay on the local machine or an internal server.

Cost. Once the model is downloaded, there is no per-token API bill for every little classification or summary.

Latency. For narrow tasks, a small model can respond quickly without a network round trip.

Customization. I can train adapters for different tasks instead of touching the original model. One adapter could be for quality reports, another for AUTOSAR questions, another for product-planning classification.

Learning. This is probably my favorite part. A small model is simple enough that I can actually experiment with the entire pipeline on my own laptop.

The project flow

The complete project now looks like this:

flowchart TB
    A[SmolLM2-360M-Instruct] --> B[JSONL training examples]
    B --> C[Validation + duplicate checks]
    C --> D[Tokenizer + chat template]
    D --> E[Train / validation / test split]
    E --> F[LoRA fine-tuning]
    F --> G[Adapter checkpoints]
    G --> H[Local chat]
    H --> I[Compare original vs trained]

And I also built a separate ~5M parameter Transformer from scratch just to understand the internals: byte tokenizer, embeddings, Q/K/V, causal attention, feed-forward layers, logits, cross-entropy, backpropagation, and generation. That model is terrible at language, which is expected, but seeing the tensors move through my own code made the Hugging Face model much less mysterious.

Final thoughts

This was supposed to be a simple "fine-tune a small LLM" weekend-style experiment, but the failed versions were probably more educational than the successful one.

At first I thought: give the model 50 facts and it will remember 50 facts. Nope.

What actually mattered was how I wrote the data, how often important relationships appeared, and how much information I tried to pack into one answer. Once I changed the dataset from broad biography paragraphs into repeated atomic facts, the tiny model finally moved from inventing a Malaysian politician to correctly knowing my actual work background.

It is still only a 360M model. It still mixes things up. It is definitely not magically intelligent because I trained it for five epochs.

But that is exactly why I like the result. The experiment is small enough that I can see the limits clearly. I know what data went in, I can inspect every checkpoint, and I can run both the original and trained models side by side on my own laptop.

For me, that is much more interesting than only calling an API and never seeing what happens in between.