Giving a chatbot a knowledge base is not the same as training a model. In this quick demo, I fine-tune a small model on my own text, entirely on my own hardware, and show the before and after.
What you'll see:
– The difference between a knowledge base (temporary access to your documents) and training (teaching the model itself)
– A 120B model staying loaded on the RTX 3060 while training runs on a different node, an Apple M4 Mac mini
– A 0.5B model hallucinating an answer about RAMDeck before training
– A 100-iteration training run, started from the RAMDeck training tab, with live progress and loss
– Downloading the trained adapter
– The same question asked again, and a much better answer
A note on scale: this is a deliberately tiny demo (a 0.5B model and a small text file) to show the pipeline end to end. Training here produces an adapter, a small file you apply to the base model. For facts that change often, a knowledge base is usually the better tool; training is for teaching style, format or domain behavior.
RAMDeck pools the RAM and compute of the devices you already own (PCs, Macs, mini PCs and phones) so you can run, and now train, models too big for any single machine.
Previous videos:
– Knowledge base demo: https://youtu.be/H29ASZhpTKQ
– 120B model across 6 devices: https://youtu.be/ok3nYjxhc1w
Source code: https://github.com/trademav/ramdeck-core-public
RAMDeck — Pool Your RAM. Run Bigger AI.
Run and train models too big for any single machine, using the PCs, Macs, mini PCs and phones you already own.
Leave a Reply