What is LM Studio?
LM Studio is user-friendly and available in binary format for Windows and Mac (a Linux version is in development). It supports many models compatible with the ggml tensor library from the llama.cpp project and requires 16GB of RAM.
Discover how easy and convenient LM Studio is to use!
What is LM Studio?
LM Studio is a cross-platform desktop application that allows you to search, create, and test various LLMs. It offers a user-friendly interface and a wide range of features to cater to both beginners and advanced users.
Key Features of LM Studio
- User Interface:
- LM Studio presents a , like interface that makes it easy to interact with different models.
- The interface includes options to explore models, load them locally, and interact with them using a chat-based format.
- Explore and choose models:
- The "Discover" section allows you to explore various LLMs from the Hugging Face archive.
- You can search for models based on specific criteria and view their details, including number of parameters, architecture, and author.
- Load and test the model:
- Once you select a model, you can download and load the model locally.
- The app provides a convenient way to switch between different models and test their capabilities.
- System requirements and customization:
- You can define system requirements to impact the output of the models.
- Customize other parameters like response length, stop sequence, etc.
- Runtime model:
- LM Studio supports various runtimes that can enhance the performance of models.
- The application provides information about compatible runtimes, allowing you to download and install them.
- Developer mode:
- For advanced users, developer mode offers additional features and settings.
- Explore options like server configuration, API endpoints, and logging.
Use models from the Chat dashboard
Once installed, LM Studio facilitates downloading models from the Hugging Face Hub, including preset options.
For example, we can download the Zephyr 7B β model, adapted by TheBloke to the llama.cpp GGUF format.
What is LM Studio? Picture 1
Activating and loading models into LM Studio is simple.
What is LM Studio? Picture 2
You can then start using the model right from the Chat dashboard, no internet connection required.
What is LM Studio? Picture 3
The right panel displays and allows modification of the default presets for the model. Memory usage and useful inference metrics are displayed in the window title and below the Chat panel.
Other models, such as the Instruct 7B codellama, are also available for download and use.
What is LM Studio? Picture 4
LM Studio also highlights new models and releases from Hugging Face, making it an invaluable tool for exploring and testing out the latest releases.
Access models using API
One notable feature of LM Studio is the ability to create Local Inference Servers with just one click.
What is LM Studio? Picture 5
The Automatic Prompt Formatting option simplifies building prompts to match the expected format of the model. The exposed API matches OpenAI format.
Here is an example of how to call an endpoint using CURL:
curl http://localhost:1234/v1/chat/completions -H "Content-Type: application/json" -d '{ "messages": [ { "role": "system", "content": "You are an AI assistant answering Tech questions" }, { "role": "user", "content": "What is Java?" } ], "temperature": 0.7, "max_tokens": -1, "stream": false }'
Response providing requested information:
{ "id": "chatcmpl-iyvpdtqs1qzlv6jqkmdt9", "object": "chat.completion", "created": 1699806651, "model": "~/.cache/lm-studio/models/TheBloke/zephyr-7B-beta-GGUF/zephyr-7b-beta.Q4_K_S.gguf", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Java is a high-level, object-oriented programming language that was first released by Sun Microsystems in 1995. It is now owned by Oracle Corporation. Java is designed to be platform independent, meaning that it can run on any operating system that has a Java Virtual Machine (JVM) installed. Java's primary applications are in the development of desktop applications, web applications, and mobile apps using frameworks such as Android Studio, Spring Boot, and Apache Struts. Its syntax is similar to C++, but with added features for object-oriented programming and memory management that make it easier to learn and use than C++. Java's popularity is due in part to its extensive library of pre-written code (known as the Java Class Library) which makes development faster and more efficient." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 0, "completion_tokens": 166, "total_tokens": 166 } }
This feature helps a lot in testing integrations with frontends like chatbots or workflow solutions like Flowise.
While not open source, LM Studio is a powerful addition to your local toolkit, allowing you to easily experiment with and apply models from Hugging Face. Its user-friendly interface and rich features make it an essential resource for anyone looking to explore the world of large-scale language models.
You should read it
- 6 notable differences between Mac Studio and Mac Pro
- Roblox Studio
- Dell is about to release Studio XPS 15 and 17 inches
- How to install OBS Studio in Ubuntu
- Shortcuts in Camtasia Studio
- Creator Studio - Facebook page management application
- What is Android Studio?
- How to adjust video audio on Camtasia Studio
- Visual Studio Community - Download Visual Studio Community here.
- Microsoft launches Visual Studio 2022
- Don't buy a Mac Studio in 2024, the Mac mini M4 is much better!
- Experience the Microsoft Surface Studio All-in-one, rival of the iMac