AI Hackathon Insights: My Personal Learning Journey

On August 28th and 29th, my company, Kongsberg Digital, hosted a company-wide AI Hackathon, open to all employees regardless of their skillset or area of expertise. The theme was to create something meaningful using AI, with the broader goal of helping every employee learn and prepare for the future of the industry. To support this, the company provided foundational AI/ML training, hosted bootcamps, and arranged expert sessions to guide our learning. A total of 21 teams, comprising over 200 participants, took part in the event, making it a tremendous success.

However, in this blog, I won’t be focusing on the main event. Instead, I’ll share insights into the personal learning journey and exploration I undertook leading up to the event.

What I knew about AI

Before diving into my exploration, I was familiar with a few popular GenAI chatbots like ChatGPT (OpenAI), Gemini (Google AI), and Copilot (Microsoft and GitHub), as well as some LLM¹ models such as Llama, Gemma, and Mistral. I also knew about a few AI-based tools for QAs. However, beyond that, I had little knowledge of how to effectively use AI or how to build tools and chatbots for specific use cases.

What I Explored/Learned

I had the opportunity to delve deeper into AI, learning how to develop my own RAG (Retrieval Augmented Generation) model².

I learned how to host local LLM models, train the model using own data sources, and the query retrieval using custom prompts.

Tools & Technologies explored/used

1. Ollama³
2. Python
3. LangChain⁴
4. Chroma DB⁵
5. Gradio⁶ Chatbot

Prior Knowledge/Skills

1. Basic programming knowledge
2. Basics of Python programming language
3. Basics of AI
4. Any IDE (Integrated Development Environment) of your choice

Ollama

Ollama is a software platform designed to simplify the process of running open-source LLMs on your local computer.

To install ollama on your machine, follow below steps.

Visit https://ollama.com/

Click the Download button on the Home Page. Below page will be displayed.

Select the OS (I have used Windows) and proceed with download.

Once the OllamaSetup file is downloaded, continue with installing Ollama on the machine by following the prompts on installation wizard.

After ollama is installed and running on the machine, user can see an icon as below in the running icons tray in task bar.

Now that the ollama is installed and running on the machine, lets understand how to get the required LLM model running on the machine via ollama.

To get the list of all the LLM models ollama supports, click on ‘Models’ option on top bar on ollama site. Below screen will be displayed with all the supported LLM models and their brief details.

Click on the required LLM model. This will display more details and the command to download/install the model. Below is the example of Llama3.1 model page.

User can select the trained token size (based on requirement) of the model from the dropdown and copy the command highlighted in above screen. Run the copied command in the Command Prompt. A screen like below will be displayed and will also display the llm model download progress (It will take some time based on the model and its size user is trying to run).

Once the model is downloaded and running, below screen will be displayed.

In above screen, user can chat with the model just like any other AI ChatBot.

With Ollama installed and the necessary LLM model running on our local machine, we can now move on to the coding phase.

The coding process will be divided into two parts. Part 1 will focus on data embedding and storage, essentially the data training phase. Part 2 will cover Retrieval Augmented Generation (RAG), where results are generated based on user query input.

Data Embedding and Storage

This module involves below steps.

1. Read data from data source (In my example, PDF file)
2. Chunk above read data
3. Embed above chunked data
4. Store the embedded data in vector DB (Chroma DB)

Assuming the user has basic Python programming knowledge and has already set up a virtual environment for Python scripting, the next step is to create a Python file (.py) for the model training script.

Run the command “pip install pymupdf langchain langchain_community” to install required python packages and copy below code in to the newly created .py file.

import fitz  # PyMuPDF
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.vectorstores import Chroma
# Function to read PDF and extract text
def read_pdf(pdf_path):
    doc = fitz.open(pdf_path)
    text = ""
    for page in doc:
        text += page.get_text()
    return text
# Function to chunk text
def chunk_text(text, chunk_size=200, chunk_overlap=50):
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap
    )
    return text_splitter.split_text(text)
# Function to embed text chunks and store in ChromaDB
def embed_and_store(chunks, chroma_path='chroma_db'):
    embeddings = OllamaEmbeddings(model="llama3.1")
    Chroma.from_texts(chunks, embedding=embeddings, persist_directory=chroma_path)
# Main function to process PDF
def process_pdf(file_path):
    text = read_pdf(file_path)
    chunks = chunk_text(text)
    embed_and_store(chunks)
process_pdf(<path_to_pdf>);

Code Explanation

The function read_pdf will read and combine texts from each page of the pdf file to a single string. This function requires path of the pdf file as input parameter and returns the combined text string.

The function chunk_text will split the provided text in to smaller sub-strings based on chunk_size & chunk_overlap value provided. This function requires 3 input parameters (text, chunk_size, chunk_overlap) and returns a list of splitted sub-strings from text. The value of chunk_size defines the size of each chunk and chunk_overlap defines the overlap size between each consecutive chunks.

The function embed_and_store will embed the chunked texts to vector form and store them in persistent vector db (Chroma DB). This function requires 2 input parameters (chunks, vector db path). This function will generate a list of embedded vectors for provided list of chunks and then stores them in the persistent chroma db.

The function process_pdf is the main function which accepts path of the pdf file as input parameter and calls each of the above functions sequencially to process the pdf to convert and store it in vector db.

Replace <path_to_pdf> with the actual path to your pdf file.

NOTE: By using a persistent vector db, we are esentially avoiding retraining the model with same data repeatedly

RAG & Chatbot Integration

This module involves below steps.

1. Read the user query
2. Create a prompt templete with the user query
3. Chain the prompt template with llm model
4. Query the vector db and return the result

Create another Python file (.py) for the RAG script. Run the command “pip install langchain_chroma gradio” to install required python packages and copy below code in to the newly created .py file.

from langchain_community.embeddings import OllamaEmbeddings
from langchain_chroma import Chroma
from langchain.prompts import PromptTemplate
from langchain_community.llms import ollama
import gradio as gr
def query_chromadb(query, chroma_path):
    # Load the Chroma vector store from the specified path
    vectorstore = Chroma(persist_directory=chroma_path, embedding_function=OllamaEmbeddings(model="llama3.1"))
    # Define the LLM and prompt template
    llm = ollama.Ollama(model="llama3.1")
    template = "Based on the following query: {query}, retrieve the most relevant information."
    prompt = PromptTemplate(template=template, input_variables=["query"])
    
    # Create a LangChain LLM chain
    chain = prompt | llm
    
    # Perform the query
    results = vectorstore.search(query=query, search_type="similarity")
    for result in results:
        return chain.invoke(result.page_content)
# Integrated Gradio chatbot interface
def ask_chatbot(message, history):
    user_query = message
    yield query_chromadb(user_query, 'chroma_db')
gr.ChatInterface(ask_chatbot).launch()

Code Explanation

The function query_chromadb will query the chroma db based on the user query. This function requires 2 input parameters (query, vector db path) and returns the result from the vector db based on the query, llm model and prompt template.

The function ask_chatbot will accept the user query (from gradio chat interface) as the input; and pass the same to the function query_chromadb to get the result from vector db. And then this result will be displayed to the user on the gradio chat interface as the response to the query.

The line “gr.ChatInterface(ask_chatbot).launch()” will launch the gradio chat interface on our local port.

Once the user runs this file, an output like below will be displayed on terminal.

This indicates that the gradio chat interface is running at local url http://127.0.0.1:7860.

Now user can access this url in the browser, which would look like below.

Now the user can start interacting with above chat interface just like any other chatbot.

NOTE: I have used the most basic prompt “Based on the following query: {query}, retrieve the most relevant information.”. However the user should write their own prompt based on their requirement.

In this blog I have used only open source tools and technologies and hosted everything on local machine to avoid data security issues. I have tried to explain the step by step implementation of a basic training and RAG model. Feel free to play around with and enhance this model to suit your needs.

The next blog will be on the actual solution we implemented during the hackathon main event. Stay tuned!

Footnotes

1. Large Language Model is a computational model capable of language generation or other natural language processing tasks. ↩︎
2. RAG is an AI framework that combines retrieval and generation models to help large language models (LLMs) produce more accurate and up-to-date information. ↩︎
3. Visit https://ollama.com/ for more information on ollama. ↩︎
4. Visit https://www.langchain.com/ for more information on LangChain ↩︎
5. Visit https://www.trychroma.com/ for more information on Chroma DB. ↩︎
6. Visit https://www.gradio.app/ for more information on Gradio. ↩︎