Building on my previous blog post, AI Hackathon Insights: My Personal Learning Journey, I’m now sharing which technologies we chose to use and the reasons behind our choices.
After evaluating various tools and technologies and considering that we had only one day to develop a functional AI solution, we chose to incorporate Langflow into our project.
With the rising adoption of large language models (LLMs) like GPT-4, BERT, and others, there is a growing need for efficient tools to manage and orchestrate these models. This is where Langflow comes in, providing a seamless experience for building, deploying, and managing LLM workflows. In this post, we’ll explore what Langflow is, highlight its key features, and demonstrate how you can easily create powerful workflows using it.
What is Langflow?
Langflow is an open-source tool designed for building, orchestrating, and scaling language model workflows. At its core, Langflow provides a graphical interface for constructing LLM pipelines, making it easy to drag and drop components, connect different model functions, and configure parameters without needing to write extensive code.
For developers and data scientists, Langflow acts as a bridge between raw machine learning models and practical applications, allowing you to efficiently manage LLM pipelines with flexibility and control. The platform simplifies LLM-based application development and brings modularity, making it ideal for both research and production.
Why Use Langflow?
When working with LLMs, there’s often more than just calling a model and getting a response. You may need to preprocess data, chain models, integrate APIs, and fine-tune hyperparameters. Langflow simplifies these tasks in an intuitive way. Here are some key reasons to consider it:
- No Code to Low Code: Langflow features a visual drag-and-drop interface that enables you to create workflows with minimal coding, making it accessible for beginners and teams looking to prototype quickly.
- Seamless Integration: It supports various LLMs and integrates with popular platforms like Hugging Face and OpenAI, allowing for easy model switching and experimentation.
- Modularity: Break your project into individual components, such as tokenization and model inference, making it easier to manage and scale your pipeline.
- Customizability: Advanced users can add custom Python scripts and unique business logic, offering flexibility despite being a low-code tool.
- Scalability: Langflow can grow with your needs, whether for academic projects or large applications, featuring caching, parallelization, and distributed computing support.
- Visualization and Debugging: Its visual approach helps identify bottlenecks and understand data flow, allowing for real-time monitoring and early issue detection.
- Deployment Support: Once your workflow is built, Langflow supports deployment in production environments, letting you expose workflows as APIs for easy integration.
Key Features of Langflow
- Drag-and-Drop Workflow Builder: Langflow’s intuitive interface lets users easily drag and drop model components, preprocessing blocks, and postprocessing actions onto a canvas, enabling the creation of complex pipelines without the steep learning curve of traditional coding.
- Node-Based Execution: Each task in Langflow is represented as a node, capable of querying LLMs, formatting data, or calling APIs. Nodes can be connected sequentially, allowing for clear tracing of information flow and logic in the workflow.
- Support for Multiple Models and APIs: Langflow supports various language models, including OpenAI’s GPT series and Hugging Face transformers, enabling easy model swapping and performance comparisons. It also allows for integration of third-party APIs directly into your pipeline.
- Custom Scripting: For users needing specific functionality, Langflow provides nodes for custom Python scripting, allowing the addition of transformations, data handling, or business logic within the workflow.
- Automated Deployment Options: Langflow simplifies deployment by enabling you to export your workflow as a web API or containerized service with just a few clicks, facilitating integration into larger systems.
- Monitoring and Logging: Langflow includes real-time monitoring tools that track workflow performance, providing metrics, logs, and error reports for each node to aid in debugging and optimization.
How to setup langflow?
Refer to the official Langflow documentation here or instructions on installing and running Langflow locally.
Step 1: Install Langflow
Install Langflow with pip on command prompt:
python -m pip install langflow -U
Install Langflow with pipx on cmmand prompt:
pipx install langflow --python python3.10 --fetch-missing-python
Pipx can fetch the missing Python version for you with --fetch-missing-python, but you can also install the Python version manually. Use --force-reinstall to ensure you have the latest version of Langflow and its dependencies.
Step 2: Run Langflow
To run Langflow, run the following command in commad prompt.
python -m langflow run
You will get an output like below.
This indicates that Langflow is running locally on port 7860. Open this URL in your browser.
You’re now ready to start building your workflow with Langflow.
Building a Sample Workflow with Langflow
To showcase Langflow’s potential, let’s go through a simple example of building a data ingestion pipeline. This pipeline will use your own data sources with Ollama to train a locally hosted model.
Step 1: Create a New Workflow
Once Langflow is launched, begin by creating a new workflow from the dashboard. A blank canvas will appear where you can start adding nodes.
Step 2: Add a Data component
Drag a Data component onto the canvas. This node will serve as the data source for training the model. In this example, we’ll use a simple File node. Enter the file path in the Path field.
Step 3: Add a Text Splitter component
Next, drag a Text Splitter component (e.g., Recursive Character Text Splitter) onto the canvas. Set the Chunk Size and Chunk Overlap values (refer to my previous blog for details on these parameters). Connect the File node as input to the Recursive Character Text Splitter node.
Step 4: Add an Embeddings component
Drag an Embeddings component onto the canvas, and for this example, select Ollama Embeddings. Specify the Ollama Model for embedding (e.g., llama3.1, nomic-embed-text) and enter the Ollama Base URL (the local URL where Ollama is running).
Step 5: Add a Vector Stores component
Finally, drag a Vector Stores component onto the canvas. In this example, we’ll use Chroma DB as the vector store. Enter values for the Collection Name (the grouping mechanism for embeddings, documents, and metadata) and the Persist Directory (where data will be saved locally). Leave the Search Query field blank, as this flow is only for training the model. Next, connect the Recursive Character Text Splitter node as the Ingest Data node and the Ollama Embeddings node as the Embedding node to the Chroma DB node.
Summary of above Workflow
The data ingestion flow is now ready. This flow will read data from the specified data source, split the text into smaller chunks, embed these chunks as vectors, and store the vectors in the vector database.
As demonstrated above, all the steps involve simply dragging and dropping components and configuring the necessary fields on the design canvas. We didn’t have to write a single line of code ourselves. This user-friendly approach allows anyone with limited coding skills to design AI workflows easily.
Advanced Features
Langflow offers many advanced features for developers to enhance their understanding and use of the tool effectively. Let’s explore a few of these features using the workflow outlined above.
Edit Code
If a developer with strong coding knowledge wants to understand the technical workings of each component, they can click on the component and then select the “Code” icon (<>) or press the “Space” button on the keyboard.
This will open an Edit Code window displaying the Python code. Developers can review the code to gain a better understanding of the component. Additionally, they can modify the code to meet specific requirements and click the ‘Check & Save’ button to apply the changes to the component locally.
Advanced Settings
Each component features an Advanced Settings button (Ctrl+Shift+A) that offers additional details about the component.
This opens a screen that provides a clear description of each field in the selected component, allowing you to enter or edit values for these fields.
All Advanced Features
Developers can see and access all the advanced features using All button (…).
Playground
Langflow offers a “Playground” option that allows developers to evaluate their AI solutions.
The Playground features a Langflow Chat interface that enables developers to monitor user input, AI output, and chat memories. This allows them to assess whether their AI is functioning as intended or if there are any issues, such as hallucinations.
API
Langflow also offers an “API” option that allows developers to integrate the designed workflow into their external application code.
This option open an API windows with complete API details of the designed workflow.
This window also provides the code snippets in different coding/scripting language formats for easy integration.
Developer can modify the field values for all components of the flow in this window under ‘Tweaks’ section.
Conclusion
Langflow signifies a transformative advancement in how developers and researchers manage LLM workflows. With its visual interface, modular architecture, and strong deployment capabilities, it serves as an invaluable tool for those looking to leverage the power of language models without the burden of complex codebases.
Whether you’re creating a chatbot, a recommendation system, or any other LLM-driven application, Langflow streamlines the process, enabling you to concentrate on innovation rather than the intricacies of model orchestration.