How to use Nvidia's AI Chat With RTX chatbot on your computer

All you need is an Nvidia RTX GPU and you're ready to start using Nvidia's new AI chatbot.

What is Nvidia Chat with RXT?

Nvidia Chat with RTX is AI software that allows you to run large language models (LLM) locally on your computer. So, instead of going online to use an AI chatbot like ChatGPT, you can use Chat with RTX offline whenever you want.

Chat with RXT uses TensorRT-LLM, RTX acceleration, and quantized Mistral 7-B LLM to deliver fast performance and quality responses on par with other online AI chatbots. It also offers Retrieval-Augmented Generation (RAG) capabilities, which allows chatbots to read through your files and trigger customized responses based on the data you provide. This allows you to customize the chatbot for a more personal experience.

If you want to try Nvidia Chat with RTX, here's how to download, install, and configure it on your computer.

How to download and install Chat with RXT

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 1

Nvidia has made it much easier to run LLM locally on your computer. To run Chat with RXT, you simply download and install the app, just like you would with any other software. However, Chat with RTX has some minimum technical requirements for proper installation and use.

1. RTX 30-Series or 40-Series GPUs
2. 16GB RAM
3. 100GB free memory capacity

If your PC meets the minimum system requirements, you can go ahead and install the app.

Step 1: Download the Chat with RXT ZIP file.

Step 2: Extract the ZIP file by right-clicking and selecting a file archiving tool like 7Zip or double-clicking the file and selecting Extract All.

Step 3: Open the extracted folder and double-click setup.exe. Follow the on-screen instructions and check all the boxes during the custom installation. After clicking Next, the installer will download and install LLM as well as all dependencies.

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 2

Chat with RXT installation will take some time to complete as it downloads and installs a large amount of data. After the installation process, click Close and you're done. Now, it's time for you to try the app

How to use Nvidia Chat with RTX

While you can use Chat with RXT like a regular online AI chatbot, the article strongly recommends you check out its RAG functionality, which allows you to customize its output based on files you give access to.

Step 1: Create RAG folder

To start using RAG on Chat with RXT, create a new folder to store the files you want AI to analyze.

Once created, place your data files into the folder. The data you store can include many subjects and file types, such as documents, PDFs, text, and videos. However, you may want to limit the number of files you place in this folder so as not to affect performance. More data to search means Chat with RXT will take longer to answer specific questions (but this also depends on the hardware).

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 3

Now that your database is ready, you can set up Chat with RXT and start using it to answer your questions and concerns.

Step 2: Set up the environment

Open Chat with RXT. It will look like the image below.

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 4

In Dataset, ensure that the Folder Path option is selected. Now, click on the edit icon below (pen icon) and select the folder containing all the files you want Chat with RTX to read. You can also change the AI ​​model if there are other options (at the time of writing, only Mistral 7B).

Now you're ready to use Chat with RXT.

Step 3: Ask questions with Chat with RXT!

There are several ways to query Chat with RXT. The first is to use it as a regular AI chatbot. The author asked Chat with RXT about the benefits of using a local LLM and was satisfied with their answer. It's not too deep but accurate enough.

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 5

But since Chat with RTX has RAG capabilities, you can also use it as a personal AI assistant.

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 6

Above, the author used Chat with RTX to ask about his schedule. The data came from a PDF file containing schedules, calendars, events, tasks, etc. In this case, Chat with RTX pulled the exact calendar data from that data; You'll have to update your data files and calendar dates for features like this to work properly until integration with other apps becomes available.

There are many ways you can use Chat with RTX's RAG to your advantage. For example, you can use it to read through legal papers and come up with summaries, generate code related to a program you're developing, get bulleted highlights about a video you've reviewed. busy to see and more!

Step 4: Additional features

In addition to your local data directory, you can use Chat with RXT to analyze YouTube videos. To do so, in Dataset, change Folder Path to YouTube URL.

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 7

Copy the YouTube URL you want to analyze and paste below the drop-down menu.

How to use Nvidia's AI Chat With RTX chatbot on your computer Picture 8

RTX's YouTube video analysis is quite good and provides accurate information, so it can be useful for research, quick analysis, etc.

Jessica Tanner

Update 16 March 2024