I asked ChatGPT to tell me what books I had written, and it gave me some titles not only of books that I hadn’t written, but which didn’t exist. I asked it to write a tourist brochure on the town in which I live, and it made up details about markets and parks that don’t exist. In order to overcome its hallucinatory deficiencies and the fact that the LLM on which it was trained ends in 2021, OpenAI began rolling out plugins to its ChatGPT Plus users in March 2023. These were designed to be the eyes and ears for ChatGPT, allowing it to access more recent information and perform a number of other very specific tasks, such as searching databases, booking a flight or making a restaurant reservation.
The plugins allow users to move beyond the limitations of ChatGPT, and to extend the functionality of the program without interfering with its code. OpenAI hopes that not only will this allow ChatGPT to regain some of the ground and confidence it lost, but also by limiting access to plugins to paying subscribers give potential customers another reason to take out a subscription.
When the plugins were first announced there were fewer than 100 of them. There are now in the region of 650, probably more by the time you read this. They have been developed by third party developers to suit their particular needs and requirements, hopefully giving them access to more customers as well as raising their profile. Early plugins were produced by Expedia, Shopify, WolframAlpha and Zapier. OpenAI reviews all plugins before they are added to its collection, but it’s up to users to decide if they are prepared to trust them or not. A recent Wired article “ChatGPT has a Plugin Problem” suggested there may be some concerns over their potential security. However, I haven’t seen any stories yet that suggest any plugins have acted maliciously and done any damage, so I think it’s more of a theoretical concern that a practical one.
The only real issue that OpenAI has had with plugins is one of its own making. It produced a plugin that integrated the Bing search engine directly into ChatGPT, making it possible to search the internet in real time. However, this had the unintended result of giving people access to material behind paywalls, so it was withdrawn a few days later, and we have no idea when, or if, it will return.
Using plugins
Once you’re a Plus subscriber you’ll have access to the plugin library at no extra cost. First of all, you need to go into your settings and enable plugins. After that you can click on the GPT-4 button, and scroll down the options to Plugins. Next click on the Plugin store to gain access to the entire collection. You can choose to view them by Popular, New, All, or Installed. Finally, there is a search option, which is becoming increasingly useful given the increasing numbers of plugins.
Choose a plugin and click to install it. You can install as many as you want, but you are limited to having a total of three in operation at any one time. If you want to use a different one, you have to swop it out with one currently in operation. You may also need to register with the organisation that produced the plugin before you can use it, which is an irritating extra step, but only needs to be done once.
ChatGPT generally chooses for itself which plugin it uses to accomplish a particular task. I generally include the name of the plugin that I want it to use in the prompt itself, just to be on the safe side. Users will then see an icon to say that ChatGPT is using the particular plugin. When it’s finished you’ll see the response on the screen. You can click to open the dialogue that the Chatbot had with the plugin, where it went, and what instructions were given and executed. For example, I used the ScholarAI plugin to find scholarly articles on using Tixagevimab plus cilgavimab for preventing Covid 19. The request that was sent to the database was
"keywords": "Tixagevimab, cilgavimab, COVID-19, prevention", "query": "Tixagevimab plus cilgavimab for preventing COVID-19"
In return I got the result nicely formatted as an answer, but could also go and look at the raw response that had been returned, which said the same thing. It was much more difficult to read, so you’ll probably only do it once for curiosities’ sake.
You could make the valid point that I could just as easily have gone to the website to search open access Springer-Nature articles. However, once ChatGPT has access to the results of a search I can then ask it to manipulate that information—I can ask for a summary of a particular article, or ask for it to be re-written. I can even ask for it to be reproduced in the form of a poem were I so inclined! Plugins do more than simply reach out to a third party resource, they give ChatGPT the ability to transcend the September 2021 cut-off date, and access material on the web that is current to whatever content the plugin provides access to.
The second benefit of using plugins is that they guide ChatGPT to specific data, reducing the chances of generating inaccurate information. If I asked the free, 3.5 version for academic articles on the extent to which those drugs prevented Covid 19, I simply couldn’t trust the results, due to the very great possibility that it would confidently make up papers, references and authors that simply don’t exist. By restricting it to a trusted body of knowledge, it doesn’t have room to hallucinate, meaning that it’s not necessary to double check the information being returned, saving a considerable amount of time and headaches.
What plugins are available?
In short, a lot! A quick glance over the ‘Popular’ list shows several that illustrate the breadth and range of them. There are plugins that allow users to link ChatGPT to an online PDF file, which can then be interrogated. Other plugins will allow ChatGPT to browse dozens of webpages in a single query, generate PDF files, generate .csv files, or build charts, graphs and diagrams. Users can install plugins to search Expedia or Kayak to help plan a holiday. These search flights and give recommendations on where to stay and what to see.
There are tools to help you search the internet as well. Let me be clear: ChatGPT isn’t a search engine—that’s not what it was designed as—but some plugins overcome this limitation. YouTube transcripts can be searched. The Wikipedia plugin allows users to ask questions about its content, search for public information from the UK’s Companies House, get answers from millions of scientific papers, search for jobs, images, social networks, the U.S. census—the list is almost endless. A search for search in the list of plugins produced over 110 tools that can be used for searching various databases or types of knowledge.
Here are a few plugins that I think are of particular interest to information professionals.
AskYourPDF: Unlock the power of your PDFs!, dive into your documents, find answers, and bring information to your fingertips.
Bookworm: AI-powered personalized book recommendations, scanning countless titles to find your perfect read.
ChatWithPDF: Chat with everything from entire PDF books to Google Drive documents just by providing a link.
Chat with video: Ask questions, analysing, and parsing through YouTube videos by simply providing a YouTube video URL.
Chat With Website: Have a conversation with any website, powered by magicform.ai.
Companies in the UK: Provides financial information on UK Companies.
Gate2AI: Discover the perfect AI tools for your needs.
Image Search: Discover complimentary images to enhance your generated article or to highlight specific paragraph.
Job search UK: Get the latest job posts from the UK's top job boards including Reed, Indeed, and others.
Keymate.AI search: Search & Browse the web by using Google Search results with KeyMate.AI, your AI-powered web crawler.
Litmaps: Get help exploring the scientific literature. Find relevant papers and generate mindmaps of the literature.
Mixerbox Scholar: Free and reliable academic search engine! Find research papers and get answers in an instant!
Podcast search: This tool explores podcasts from PodcastIndex.org, a platform for decentralized audio content discovery.
Prompt Perfect: Type 'perfect' to craft the perfect prompt, every time.
Puzzle Constructor: A tool for creating crosswords. You can create crosswords from words and hints.
ScholarAI: Unleash scientific research: search 40M+ peer-reviewed papers, explore scientific PDFs, and save to reference.
Scholar Assist: Search academic research papers from arXiv and find answers to your questions.
Scholarly: An AI-powered search engine for exploring scientific literature.
Search UK Companies: Fetching public information on UK registered Companies and Officers from Companies House.
Show me: Create and edit diagrams directly in chat.
Social Search: The Social Search provides access to tweets, users, followers, images, media and more.
Stories: Create beautiful, illustrated stories easily.
There’s an AI for that: Find the right AI tools for any use case, from the world's largest database of AI tools.
Timeport: Begin an exciting journey through time, interact with unique characters, and learn history in this time-travel game.
UK Latest News: Get the latest news stories from the UK's top news outlets including BBC News, Sky News, The Independent and more.
UK Politics: Search through UK political documents such as speeches, press releases, voting records, and candidates' profiles.
Video Insights: Interact with online video platforms like Youtube or Daily Motion.
Video Summary: Summarize YouTube video highlights. Generate summaries from YouTube video URLs.
Vox Script: Enables searching of YouTube transcripts, financial data sources, and Google Search results, and more.
WebPilot: Browse & QA Webpage/PDF/Data. Generate articles, from one or more URLs.
Web requests: Goodbye Knowledge Cutoff, Hello World! This is your AI assistant's web browser. Just enter a URL
WebRewind: Get the picture of a website at a specific date.
Wolfram: Access computation, math, curated knowledge & real-time data through Wolfram|Alpha and Wolfram Language.
WordCloud: Create word cloud images from text.
WordPress Publisher: Publish content directly to a Wordpress blog.
Zapier: Interact with over 5,000+ apps like Google Sheets, Gmail, HubSpot, Salesforce, and thousands more.
This shouldn’t be taken as any kind of comprehensive or exhaustive list—new plugins are appearing almost daily, and now and then one is retired, so it would help if there was a plugin in to help with all the plugins. In actual fact there are two, MixerBox FindPlugins and Pluginpedia, both of which can identify plugins to assist you in your job.
Advanced Data Analytics
OpenAI hasn’t stopped at providing access to plugins. It also produces a tool (again subscriber only) called Code Interpreter. Which, to be fair, is one of the most boring and ineffectual product names that I have ever seen. The company obviously agreed with me, because OpenAI renamed it to Advanced Data Analytics at the end of August 2023. To be honest, I think the new name isn’t much of an improvement, although it’s slightly more descriptive as to what the plugin does. It’s an amazing tool, basically a member of staff who is trained in using analytics tools to interpret any data that you give it.
When you want to use it you simply call it up from the same pull down menu used for plugins. You can then click on the little plus sign in the search box, and upload your file, such as a .csv, excel spreadsheet, SQL database and so on, and ask ChatGPT to give you insights into what it finds. ChatGPT is excellent at explaining what exactly it’s doing and how it’s interpreting the data that it finds.
I gave it an Excel spreadsheet on a year’s worth of data on how far I walked each day, both in terms of steps, miles and timings. It was able to work out exactly what the data was, how big the dataset was, and describe the type of cells. It immediately did some calculations to work out approximately how far I walked every day in both steps and miles, and how long it took. I was then able to see which days I walked the most, average steps to a mile, average time taken to walk a mile and so on. I was then able to get it to produce a graph based on dates and steps, dates and miles, and pie charts to show the total steps taken in each month as a percentage of the total number of steps taken. If I’d also listed the different places that I had walked, it could have produced a map for me as well colour coding locations to distances walked.
This was all painfully simple data, but even so, it would have taken me a reasonable amount of time to extract that information myself. I’m sure you can come up with your own examples, but I asked ChatGPT itself for some suggestions. Text analysis—analysing customer reviews to determine overall sentiment about library services. Data management—removing duplicates or correcting inconsistences in library databases. Statistical analysis—using statistical models to predict which books or materials should be stocked based upon past usage data. Date and time information—understanding the busiest and slowest hours of the library to optimize staffing. Other examples that I’ve seen include taking data on world heritage sites and creating maps that produce a .png file that I could download, detailing sites per country. I could give it information on Pound to Dollar exchange rates and ask it to plot daily prices in a chart.
It doesn’t end there. It can run and test code, interpret it and then explain it to you. It can produce Python code to solve mathematical problems. You can upload equations and get the full answer, including all of the workings, which is going to give math teachers headaches, I’m sure. It can identify insights from the datasets that it’s given, predicting patterns and trends, provide recommendations or action items based on what it has discovered from the dataset. It can even then go ahead and draft a report for you based on the analysis of the data that you have given it.
I would go so far as to say that, for many people, this is going to be the single most useful tool that OpenAI has produced for use with ChatGPT, and we haven’t even scratched the surface of its potential. Of course, it’s very far from perfect—the file size limit for uploading data to be processed by the tool is 5MB at the moment. As a result, you may need to preprocess it locally before uploading a smaller subset. If this is an issue for you, it may be worthwhile taking a look at one of ChatGPT’s closest competitors, Claude.ai, which allows users to upload 5 files, 10 MB each, and accepts .pdf, .txt, .csv and so on. The disadvantage is that it can’t produce graphs or visualize trends, but on the other hand—it is free. If you’re not satisfied with that, you could always visit Perplexity.ai and upload a file for it to analyse for you but you’re limited to a single file up to 25MB with a total of 4 uploads a day.
The future
It would take a braver, or perhaps more foolish person than me to predict exactly where we are going next. However, that would make a pretty poor ending to an article, so as long as you don’t hold me to any of my predictions in this area, I’ll give it a go.
We are already seeing competition to ChatGPT by Bard and the aforementioned Claude and Perplexity. It’s only a matter of time before they start to offer their own range of plugins, and if they can offer them for free it’s going to put pressure on ChatGPT to make its offering available to everyone, not just the paying subscribers. By the end of the year there will be thousands of plugins available, and they will be able to work seamlessly with their associated Chatbots. While all of them will have some limitations in terms of the currency of data and the tendency to hallucinate a balanced use of plugins will alleviate those problems.
Plugins will increase in functionality. They will be able to translate conversations real time, and will be able to convert text to speech, or by adding sign language avatars. Users will be able to customise the look and feel of plugins—ChatGPT recently added in a feature to customise itself, so it’s only logical this will also happen to plugins as well.
At the moment plugins are to all intents and purposes very basic, since they act as the filling in the sandwich of data and Chatbot. I can expect them to start to work together, so if I want to go on holiday I can use one plugin to find a good location for me, then pass that information onto a second which can suggest good hotels, before handing off to another to book reservations in restaurants that serve the food that I like.
Alternatively, we will end up with all encompassing plugins that will offer to do everything for you in one go. We will see domain specific features, so I’ll be able to use a plugin to talk to my fitness device, identify my physical activity, talk directly to my physician to set up appointments for me or get repeat prescriptions while simultaneously searching for interesting articles that are pertinent to my conditions. A hypochondriac’s dream I’m sure, but you get the point.
We will have plugins that are able to work across teams. If a number of people are working on a project the plugin will be able to search for information, channel it through to the appropriate team member in detail, while sending a brief overview to others to keep them in the loop. The plugin will be able to handle discussions, create to-do lists, arrange meetings by accessing team members calendars, do real time searching during the meeting, translate conversations into different languages (real time) and display charts and graphs to the entire group based on what data is uploaded.
If we thought that ChatGPT was going to be a revolution in the world of information, it’s only going to be the tip of a very large AI iceberg, most of which will be plugins and data analysers working beneath the surface.