Software Diagnostics and Observability Institute

A Hands-On WSL2 Laboratory for Inference, Training, and Deployment, CUDA Programming, GPU Diagnostics, Observability, and Debugging with Complete Exercise Transcripts

Table of Contents: https://www.dumpanalysis.org/files/Local_AI_on_RTX_Spark_Laptops_Version...

On a 64 GB RTX Spark laptop, the WSL2 guest reports 15.70 GiB of RAM. Can a process inside that guest allocate and use 18 GiB of CUDA memory? We answer by allocating eighteen tensors, writing to every byte, and checking the values while all eighteen tensors are still live.

In Local AI on RTX Spark Laptops, we take the same approach to inference, training, and deployment. When a model is too large to fit entirely on the GPU, we read the loader log to find which layers stayed on the host. When a fine-tuned checkpoint reloads successfully but produces different continuations, we compare logits and generated tokens. When CUDA arithmetic passes, but the profiler records no usable GPU timestamps, we reduce the workload to a small reproducer.

The 251 hands-on exercises extend from writing CUDA kernels to training embedding models, investigating multimodal output, building diagnostic agents, and serving models with Docker and Kubernetes. The book includes 483 recorded command transcripts. The GitHub companion provides the programs, fixtures, and checkpoints to repeat the investigations and compare your observations with ours.

The accompanying examples are available in our SparkBook repository.

The draft version of the book is available on Leanpub.