LLMs incidence on my daily basis

- 7 mins read

The idea that made me kickstart this blog is the very recent arrival of LLMs in our life.

Disclaimer : this is just some mood bill with few tech opinions. And these opinions could be easily found in a pub after a few pints.

Gambler : Origins

I was born a gambler, I will die a gambler. But that doesn’t reward me a lot to be honest. And I can’t feed my daughter with buttered pasta everyday. So I decided to put all my efforts in a side project that consists of gathering a lot of data, and feed machine learning models like a headless chicken.

image - It’s ok girl I have a master degree in computer science.

The very beginning of this project is data collection, as an experienced Java developer I’ve wrote a lot of code, trying the new kid in the block known as Quarkus, doing all the typical industrial job with handsome generic typing, polymorphism and inheritance, all configurable with docker image generation and associated docker-compose to provision quickly the whole chain.

This server was used to clean and aggregate data in a nice RDBMS, allow asynchronous export of the whole base in CSV for machine learning tasks purpose, and offer APIs that allows to keep track of the various performance of the system and plotting some timeseries. Classic shit.

Of course, in this post, we won’t mention my experiments with ML. While not particularly technically interesting, they were amusing in some ways, resembling the invention of dynamite, minus the explosion.

Denying my origins

And I’m not talking about gambling there. Java is what I’ve always done in my life and I appreciate the deterministic nature of this language and its rich (and more or less) stable ecosystem.

After failing to beat the flip-a-coin model, I decided to completely change my “good Java student” approach that used properly sized columns, data reconciliation with foreign keys and all the usual stuff of big coropration, for a greedy approach storing as many data as I could and feed it in the ML.

image - Goodbye my lover, goodbye my friend.

This approach led me to completely overhaul the entire architecture, implementation and data model. This project made me even poorer than where I started, and it was already time to add a lot of dirty work to do.

At some point I wanted to try either Golang or Rust, I was curious about the prior one especially because it fits my daily basis tasks (backend information systems oriented), and what it could offer. Whether it’s a good choice or not will not be discussed there, let’s just keep in mind I’m now targeting superfastquick processing and I’ll be coupling this Golang with MongoDB because, fuck columns.

Fortunately, it more or less coincided with ChatGPT arrival, and it was the beginning of my journey with my one digit IQ friend Artificial Intelligence.

Speeding up the delivery

The first encounter was like meeting aliens, with unlimited knowledge. I have to admit, with hindsight, that at this time I was in an AI berserk telling everyone that singularity was there and I could finally drink piña coladas while the robot was setting up CI/CDs and industry grade code to accomplish any task.

image - My ethically acceptable slave.

After completing Golang tour, I was ready to revamp my good ol’ Java into state-of-the-art Golang. I had the feeling ChatGPT turned my learning curve of this new language from a growing function to a crazy asymptote. Every module rewriting was painless, all I had to do was copy-pasting all the source code and the outputed Go code looked beautiful and production ready. Who in this planet would need unit test when the logos machine knows everything and set it up properly, in accordance with your fine-crafted prompts ?

Maybe sometimes I wondered myself what would all these junior developers would become, or even I, but I didn’t had to worry a lot because my successful project was at my reach and soon I would fight the Zucc and Elon Musk in a cage.

I was ready to compile and launch my pinnacle of IT engineering.

Dr Jekyll & Mr Hyde(ous)

The unexpectable occured : nothing worked. Yes, I had to do some compilation fixes but I was blinded by the shiny power of LLM. The troubles started, what was wrong, was I concise enough in my prompts ? Or maybe the ChatGPT corpus was too old, too cranky, the poor model was questioned as he was hiding French resistance in his wine cave.

I needed to be sure, then I made him bootstrap me the same project in Java, a field where my perfectly trained eyes could detect any hallucinations. And this was the revelation.

image - The wheeled equivalent of the generated codebase.

The whole codebase was bullshit.

Trust was broken, this chatbot was a scam, all the code it was serving me off the shelf was a fraud. But how, it looked so confident, the code appeared to be properly writted, how can my oversized ego of developer couldn’t notice it before.

This was frustrating, but the technology still looked promising under the hood. Maybe it was time to put my vanity aside, and get my hands dirty, as in every good project.

AI Kung-Fu

Instead of being parachuted onto the summit of the Himalaya, we need to start from the bottom. No magic tool ever occured since Turing created the first IT ancestor of us, and the statu quo won’t be disturbed.

Then we had to start small. Let’s make a small snippet to accomplish a small task. And another one, and another one. With these snippets in context, assemble them.

And there, step-by-step, I re-writed the modules one by one. The beast needed to be tamed.

image - Maybe I shouldn’t have quit sport.

There was still enough space for junior developers, senior developers, developers. The model is strong, but the error is still located between the keyboard and the chair.

It was a very interesting exercise as I really re-writed the whole project in a very short time, with a code quality I judge quite acceptable. I’m now using it everyday and it fullfils what it has been made for and even works better than the original version.

Though, I had to learn to be cautious about what he was suggesting me, or in my prompts. Too much liberty / vague prompts led him to complete the job freestyle. And never challenging it’s code, even mentally, is a huge mistake.

Paving the way towards real life Jarvis

If you reached this line, congratulations, you’re reaching the end of this mess.

I’ll conclude with some insights related to this post, to step up the quality of this post.

At first we can look into this Time article about AI progress, it clearly shows that AI is not at human level for code generation.

Yet.

The curve isn’t clearly a logarithm curve, and at the time this post was written, Meta released CodeLlama and HuggingFace already put their hands on it. Theorically, it’s able to being fed a lot of tokens, so we can imagine a future where we input the whole codebase and tell it to do the magic. And still this model is being challenged by models such as WizardCoder with impressive results on benchmarks.

There are severals open-source projects that are also raising, such as GPT-Engineer and its hidden children DevOpsGPT ,or GPT-Migrate that allows migrating a codebase to another technology.

Enterprise will face the challenge to properly equip their IT teams with these LLMs without issuing data leakage and there still a long way to the real life Jarvis, fulfilling all our code fantasies.

image - Porting DOOM to the Roomba representation in Iron Man.