Episode 93: Exploring Large Language Models

Lisa Peña: Many of us use large language models or LLMs daily. LLM-powered platforms give us quick answers, show us what's possible, and help with to-do lists, all while using a conversational, human-like approach. But can LLMs take on a big challenge like advancing world-changing research and development? An SwRI team explored the large possibilities with large language models. That's next on Technology Today. 

[MUSIC PLAYING] 
 

We live with technology, science, engineering, and the results of innovative research every day. Now, let's understand it better. You're listening to the Technology Today Podcast presented by Southwest Research Institute. From deep sea to deep space, we develop solutions to benefit humankind. Transcript and photos for this episode and all episodes are available at podcast.swri.org. Share the podcast and hit that subscribe button on your favorite podcast platform.

Hello and welcome to Technology Today. I'm Lisa Peña. Maybe you've used large language model or LLM technology to write an email or plan a trip. LLMs are artificial intelligence available through apps and devices trained to follow conversational prompts and generate coherent, human-like responses. SwRI Researchers created the large language model adoption for multiple purposes or LAMP-focused research program to explore new LLM applications and establish best practices for responsible, safe, and effective use of LLMs at SwRI.

Institute engineer, Lynne Randolph, is here to give us an overview of LAMP. And senior computer scientist, Jonathan Esquibel, will share details about his exciting LLM research projects. Thank you for being here, Lynne and Jonathan.

Image

The Large Language Model Adoption for Multiple Purposes or LAMP focused research program demonstrated LLM impact across diverse domains, including space science, transportation, cybersecurity and more.

Lynne Randolph (LR): Thank you for having us. 

Jonathan Esquibel (JE): Yeah, thank you for having us. 

LP: All right, so we'll discuss LLMs and talk about the LAMP-focused research program in a bit. But first, let's start with understanding LLMs a little more. Can you tell us some ways you use LLMs in your daily lives? What are some recent day-to-day tasks that you've tackled with LLMs? And what platforms are you using? 

LR: Sure, I recently took LLMs to take our research information that we've done for internal research projects and make graphs. So the large language model wrote some code in the background, generated some graphs. 

But I also used it in my day-to-day life, not just at work, but also at home. You were talking about planning a vacation. I actually have mine doing a scheduled task to, every week, look at prices for vacation and find me when the good deals are going to be. 

LP: Yeah, that's great. 

LR: But in everyday work, we use our LLMs all the time to do things write code, summarize documents, help us review RFPs, so requests for proposals from clients, and see what we need to tell them to respond to it. So they don't take the place of what we're doing. They just help us be better at it and faster and more efficient. 

LP: And what platforms are you using? 

LR: So I use all sorts of platforms, from our FedRAMP version of OpenAI's GPT model to the public ChatGPT for my private use, to Microsoft Copilot, which I have the preview version, which will be rolling out to the Institute pretty soon, and then GitHub Copilot, which helps with coding tasks. It also can do things write proposals for you though. So it's more powerful than just writing coding tasks. 

LP: All right. So you're using LLMs for, as you said, work, for your personal tasks. And, Jonathan, shifting over to you, can you tell us a little bit about how you use LLMs in your daily life and what platforms you use? 

JE: Sure. So of course, I use OpenAI and the Institute's model for work. That's boring. But it definitely helps accelerate how fast I'm able to get things done, which is nice. The much more exciting thing is, personally, I use Claude, which I find phenomenal. All of Claude's models are very capable. My buddies actually use it to help them with their businesses in creating marketing material, helping them with Google Ads. And it's fun that I'm their AI guy they can come to whenever they have questions because it's not giving them exactly what they want. 

I also keep dart frogs. And when I was building my first terrarium, rather than going through a 20 minute YouTube video, which I did watch a video. But, it's so much faster to just have an AI tell me what are the important layers that I need to do to build my bioactive terrarium for this really niche knowledge? And it can just do it. 

These things are super fun. I write birthday rap songs and send them to my friends. They're a huge creative outlet. They're a huge productivity outlet. It's amazing what these things can do. And Claude, Open AI, Z.ai, there's a monolithic amount of these tools coming out. And there's only more to come, I'm sure. 

LR: I mean, we should mention Gemini since Google makes Gemini. And I also use Claude. And what's interesting is that they're better at different things. If I want to write a short story from information, I will go to Claude because it is more colorful, flowery, not necessarily more verbose. But it's more creative, maybe, than using ChatGPT. So you can give it the same prompt and say, write me a short story. And you will get something that's more moving and impactful to you out of Claude than you would from ChatGPT. 

LP: OK. So you guys are going to give us some tips at the end on how to ask the right questions to get the most out of our LLMs. We're looking forward to that. So it really sounds and, as most of us are coming to find out, there really aren't too many limits on LLMs. From rap songs, vacations, graphs, they do a little bit of everything. So Lynne, I'm going to go back to you. Let's go back and define large language model. We're using them. But what is an LLM? How does it work? 

LR: So I guess I'll say, artificial intelligence has been around a long time, like 1950s I think. And we've gone through different waves of when things are interesting in AI. And this new wave has been about generative AI. And generative AI is all about predicting the next thing. So when you think about a large language model and you're asking it a question, you're thinking, oh, it knows all this information. And it's like a human. And it's responding to me. 

But what it's actually doing is predicting the next word. So it's just taking what it's generating and saying, what should I say next for each word? Which is why you shouldn't always necessarily trust what the large language model gives you. 

LP: So it definitely needs that human touch. 

LR: Definitely needs a human touch. I'm going to let Jonathan add a little to this. It's one of his favorite topics. 

JE: It is one of my favorite topics. And I'll say what Lynne just described was absolutely an LLM. Rarely though, in these days, are you actually interacting with an LLM. You're normally going to be interacting with what's called an agent. And so what agents are is they're LLMs, but they have access to tools. They have access to planning. So for example, if you were to ask an LLM who is the CEO of Red Lobster, it might initially tell you a wrong answer because what it's trying to do is it's using its worldwide knowledge to say, who is the CEO? CEOs don't change often. So it assumes it can just tell you what it knows. 

However, that CEO recently changed. So you tell it, hey, that's not right. It changed. It says, oh, I got you. Don't worry. Then, it performs a web search. Behind the scenes, it's going and scraping the web, collecting data, reflecting on that data, making a plan. What is it that I even net equivalent Google to find that information, and then relaying it to you, oh, actually the CEO of Red Lobster is blah, blah, blah. And they recently just got that position. So when you're interacting with an agent, it's even more than just predicting a word. It's predicting what tools should I use in order to answer your question. 
 

As part of LAMP, SwRI created an LLM tool for journalists to quickly find information on any topic. This video shows how guided prompts result in real-time data, key findings and information gaps that signal information is missing and an investigation may be required. The platform also provides recommended follow-ups and sources, critical to maintaining journalistic integrity.

LR: And that's what makes them really powerful. So when we first started the LAMP program, they had just come into play. And the agentic part wasn't there. So it was really just the predicting the next word. And we've watched over the last two years how things have changed. And they get more powerful because of those tools that they can call in the background. And so we're just seeing AI and generative AI, LLMs get more and more powerful and more and more useful to people as we go on. 

LP: So how is it learning the words that it uses? 

LR: It was trained by their makers. And some of those will tell you what they trained on. And some of them, you're speculating what they trained on. So they're taking a large corpus of data, the internet, the internet archives, scholarly articles, Wikipedia, various information. And they're using that to train the LLM with information. And so that's the information that it can pull from if it doesn't have access to any tools. 

LP: What are some advantages of using LLMs? We've discussed quite a few already. But if you had to name some of the top advantages. 

LR: I'd say the top advantage is improved productivity and efficiency. So if you can generate prompts properly and give the AI the right data to give you information, it can accelerate what you can do. So I'm really a good writer. I like to write. I write proposals all the time for our division. And I'm pretty fast at it. But an LLM can help me be even faster. 

JE: I would say it's the accessibility of interface. So instead of having to learn some fancy new software tool, learn a new programming language, or do any of those things, this acts as an interface. You just type what you want. And what you want comes out of it. Now, it does take a little bit of knowledge to double check what the LLM is providing you. But some of my buddies that are making websites for their businesses or interacting with Google marketing, they never, in a million years, would be able to do that on their own without a monolithic amount of training. But because they can just ask Claude, it happens immediately. 

And there's some implications to this. If you haven't heard, Anthropic had a model called Mythos that the government stepped in and shut down. Even their neutered version, called Fable, which was supposed to have all of the cybersecurity stripped out of it, and there's a reason for that. It's because of how easy it is to access things. It makes it that these things are actually dangerous as well. Because suddenly people that don't have the experience can start utilizing more complex technologies like the ones involved in cybersecurity. So there's a lot here. 

LP: Yeah, so I think that gets us into our next question nicely. What are some of those disadvantages and other challenges when using LLMs? 

LR: Well, I think there's a couple of basic challenges. One is that, as a person using it, if you don't know what you're doing — so if you don't have knowledge of the domain that you're doing something in and you don't check your data, then you're getting information that you don't know is true. But you may be acting like you think it's absolutely true. Because an LLM will sound super confident when it gives you information. And you have to know if you need to check that information or not. 

And so when we're talking about these people who are writing code without ever having written code in the background, they may be writing code. But that code may not be great. You may look at it. It may be fragile. It may be not maintainable. And they don't know that because they don't know how to write code. 

LP: Let's get into LAMP a little bit more. Tell us about SwRI's large language model adoption for multiple purposes. It is a focused research program called LAMP. And what was the purpose of this research? 

LR: So we saw LLMs exploding out there and people using them and not a lot of information about what were the best ways to use them? And how are they fragile? Or what do we need to know about them in order to help our customers do work with them? Because everyone wanted to use large language models. They were everywhere. And we said, oh, all right. How can we best get our hands around this and get the right information we need to do our work and help our customers? 

And what we did was we thought, well, if we do this focused research program, which is not something that Southwest Research Institute does frequently, but occasionally we see a disruptive technology out there. And we say, oh, we really need to have something where, across the entire Institute, we're doing research in that area in order to further our knowledge and the world's knowledge in this area. 

And so what we did was we said, all right, let's put together a program of a bunch of people from the advisory committee for research. So we put together a committee. And what we wanted to do was make sure we did things in a safe manner. But we exposed this to the entire Institute. And the way we did that was make a program that the barriers for entry were very low. So the first thing that people had to do was put together a presentation that was kind of a quad chart, which is very specific for government usually does. 

What is my research I want to do? What do I expect to get out of it? What are the benefits? And because we made that very easy to get into, we had representation from all of the technical divisions, but also our non-technical divisions were able to submit proposals for this as well. 

And one of those was a project for the library, which was very interesting for all of us because they wanted to make our internal research projects accessible to everyone across the Institute. Right now, it's hard to know what we've done in the past for internal research. This makes a searchable database for everyone to be able to see what we've done in the past. 

So as part of that, then we put together those proposals, had people come and present on their proposals, down selected, ran some projects for a short period, like four months. And then after those projects were done, we evaluated the success. We made them submit proposals if they wanted to go into a longer phase. And we ran some longer year-long projects as well. And we did two rounds of those. So we're wrapping up, actually, this month in the LAMP-focused research program. 

LP: OK. So for the first round of research, you had 22 proposals submitted, 10 were chosen to move forward. So this was the first phase or the discovery phase. Then, of those 10, 4 moved on to that next phase. But there was also a more focused phase where you started with 9, then went to 3, then went to 2. So how did you determine which projects to explore? 
 

Image

SwRI Institute Engineer and LAMP Program Lead Lynne Randolph, pictured here presenting on LAMP, is an artificial intelligence expert supporting traffic management systems, connected vehicle technology and cybersecurity risk management projects.

LR: Well, it was interesting because we did get a wide range of proposals and different types of things they wanted to explore. And during that first discovery phase, we actually allowed multiple projects to explore the same type of thing. So one of the base things that LLMs can do is you can put a set of documents behind them. And you can have someone be able to query those documents. 

And that's a functionality that's very useful across the board. We let multiple projects explore that kind of technology so that we were able to see what the best practices are, learn lessons, and do kind of cross pollination across those projects. 

And then, when we went to the more focused phase, we were more looking for what kinds of things do we still need to research about the LLMs in order to progress ourselves? And so we went to more focused things like, can we make the AI explainable with source information? What other niche technologies can we do? One of them is looking at how we can generate chemicals from large language models. 

So more niche things, but also more wide ranging. That explainability is really important because right now the large language model is very black box to us. We put data in. We get data out. We don't necessarily have a good feel for how good that data is, how it's generating that data. 

LP: Tell us more about the diverse LAMP project domains. There were several. 

LR: Sure, we did projects in transportation. We did projects with documentation, coding. We did a robotics project on whether you could help the people move robotics without needing to know programming language behind it. So also space and then defense and safety. So all the basic categories that we do at the Institute were covered. 

LP: All right, Jonathan, I want to turn the conversation to you and talk more about your projects that were chosen to demonstrate LLM possibilities at the Institute. So first, why did you want to be part of this LAMP-focused research initiative? 

JE: Sure. So when I got my master's degree prior to the Institute, I had a specialty in Twitter sentiment analytics. 

LP: Explain that. 

JE: Yeah, this is quite fun. Because back then and this was in 2018. What I was doing I was deploying bots to autonomously scrape Twitter and detect sentiment changes. So the interesting thing was there was a shooting at a church nearby. And so the sentiment spike actually showed up on Twitter prior to EMS actually being alerted that there was something going on. So Twitter sentiment analytics is a branch of natural language processing. Large language models has pretty much outright replaced a lot of how natural language processing initially went. 

And this is kind of difficult because large language models are generally more expensive and more taxing computationally. And I'm immediately interested in LLMs just in general because I kind of want to show, well, these old techniques are still useful. They have their specialties whenever you don't need a transformer to tell you, oh, how do I set up a dart frog tank, also write me code, and do all these million things. Sometimes there is a special use case. 

Also, Twitter has been completely replaced as well. So it's fun, seeing how I'm not that old, but everything that I've done seems to be advancing quickly. 

LP: All right. So with a background in the sentiment analytics, you kind of fell naturally into LLMs. That's the next evolution of what you were studying your field. So let's talk about your projects. You found LLM capabilities are useful in traffic management, especially in safety scenarios. So tell us about your transportation research and results. 

JE: Sure. So with the transportation technology, this goes towards the idea that interfacing is one of the most powerful things these large language models can do. In the transportation industry, the operators that are working in some large center for management, they tend to have some high turnover. And they don't necessarily have a ton of training time to get on the job and start handling all these events. 

So the intention with the technology was to be able to create an interface to sort of short circuit the need of having a lot of training time, because rather than figuring out where do I need to click in this interface? You just type it to a chatbot. And it just pulls up the interfaces for you. So we still keep the human, ultimately, in the loop, but we just make it easier for them to get these things done. It also is able to aggregate a lot of data all at the same time. 

So if you wanted to ask a generic question about, well, tell me about crash incidents in the past month, where were they? You can query a database, perform the retrieval document search. This is called RAG, by the way, perform a RAG search in order to give you an answer. 

LP: So I noticed in a demo that you sent me, you could pull up a specific intersection, a camera. And tell us how that could be useful. 

JE: The use case really came from Lynne and her team with working with our transportation software. 

LR: So one of the things that is most difficult when a transportation operator is trying to figure something out is people call in with something happened at this intersection. And if you don't know the geography of wherever you're working or you don't know where that specific intersection is, it makes it difficult for you to find the appropriate devices, like the camera, to go look at it. 

So what Jonathan did was make it where someone could say, here, this happened here. And it would pull up the closest camera to wherever it was. So again, we're trying to make it easier for those operators to do their job. No one wants to replace the operators. They're very important to what they do. 

What we want them to be able to do is be more efficient at what they do, be able to accelerate their finding of incidents on the road, and reduce the response times. So get people out there to clear up the accident and get EMS on the scene quicker. And that's a way to accelerate those things. 

LP: OK. And, Jonathan, another project you're working on, and LLM dashboard for newsroom assignment editors to sort through story ideas. Great tool for journalists. Tell us about this one. 
 

Image

SwRI Senior Computer Scientist and LAMP Program Researcher Jonathan Esquivel presents at a recent LAMP briefing meeting. Esquivel is experienced in machine learning, deep learning, artificial intelligence, computer vision, data science and data mining. He developed two projects for LAMP Program exploration.

JE: Sure. So a lot of it started with that the current technologies for looking up information on the web. So, think of your public ones like OpenAI, just a free account, or Claude or Gemini, which forces you to look at its results. They do have citations. But those citations were actually found to not always be accurate. Sometimes the links that they provide, if you were to actually click on them, go to dead pages. Or they go to pages from a decade ago. And it's no longer relevant to the topic that you're addressing. 

And so for a journalist, this is harmful because you might be reporting completely inaccurate information with completely inaccurate citations. So the goal was really to how can we provide some level of explainability? And when we say explainability, this isn't me explaining how the AI works, but more so showing its work. What did you actually leverage in the documents that you collected that you were using to supply this answer? 

And so we actually used the National institution of Science and Technology, NIST. They have a risk management framework for generative AI applications, which we use that as a foundation. It was actually kind of interesting because as I was developing this at SwRI, NIST was also developing and advancing their own risk management framework. 

I sat in a class and learned that they are doing a similar approach, where they have sort of an auditing process where smaller-scoped LLM agents are auditing every single step that the major agent takes. That way, you can guarantee citations. Direct quoting is actually happening rather than just trusting. Oh, I'm pretty sure it's using its good judgment. 

LP: So this is about news teams having reliable sources for their information. 

JE: That's right. And it also enables the news teams to better prioritize what it is that they actually want to look at. Because for example, if you were to ask a question about how is the San Antonio airport doing? Is there a lot of delays right now? And you were able to get immediate information, maybe that doesn't mean that you want to report it because it's something that anybody could access. 

But if you ask a question and there's not really online material yet that's able to immediately answer it, the basic who, what, when, where, why sort of framework, that might indicate, OK, maybe we want to go investigate this. So not only is it a tool to better understand your citations, but it's also, sort of through what do you not find? Able to help journalists prioritize what they want to look at. 

LP: Yeah, as a former journalist, I can see how that could save time and be of value. The next question that comes up with these projects and I know is a concern is could LLMs and these amazing tools replace workers? You mentioned, that's not the goal. That's not the intent. But that is a concern. So how do you address those concerns? 

LR: Over the years, we've had new technology come in that people have been afraid that would take their jobs. We think about Henry Ford and the assembly line. Everyone was like, oh, we're going to lose our jobs. Or even back to the old fable where they were hammering in stakes in the ground, and the one guy was like, no, I need to do it and killed himself hammering the railroad stake into the ground when there was a machine to do it. 

What typically happens is the workers need to be educated and reskilled in order to work with the technology. So what happens is their job isn't replaced. Their job apparently migrates to something slightly different. So if you think about coding, so I'm a computer engineer by background. So what I like to do is write code. I don't want to be replaced. I still would like to write code. 

What I find is that it makes me write code faster. It helps me look for problems in the code that maybe I didn't see. So say you have something deployed and you're like, oh, I'm seeing some latency where it's taking too long to do whatever. I can ask it to look for that information in the code and point me to where the problem is. So it's making my job easier, changing the way I do my job, but not replacing me as a person. 

JE: I think a lot of it is about perspective and culture that we're still sort of adapting. There's been numerous companies that have announced huge layoffs of coders specific in that why do I need a junior coder when I have AI that does the job better, faster, cheaper, and more consistent? And honestly, there are some engineers that I'm sure that I have interacted with where that it might be an accurate statement. 

But largely, can you replace the human intuition or creativity or planning? No. One of my favorite use cases actually came out of me interacting with the Society of Motion Picture for Television and Engineers, SMPTE. They had a fully featured AI-developed movie. And you see that. And you say, wow, the people that are in charge of the creative industry are making AI movies. How absurd. Are they trying to lose all their jobs? No. 

What they actually did in the development of this movie was they tried to figure out, how do we work with AI in order to make these things happen? So you take something like concept art or storyboarding. These are plugins that they use in order to kickstart their process. They were telling me that it took maybe 200 to 300 prompts for five seconds of video. So in terms of being competitive, it's not necessarily competitive yet for them to start worrying. 

But moreover, the process that they actually developed was how do we augment our pipeline to plan for these tools to be used? So when we talk about I think Lynne had a great analogy of comparing it to technology in the past. It's kind of like the advent of the internet. There were tons of salesman jobs whose lives were completely transformed when I didn't need to buy a vacuum cleaner from a guy coming to my house knocking on my door. 

No, I could go online and I can go on Amazon and buy 100 different vacuum cleaners. We as a group and culture just need to sort of catch up to speed. And there'll be some growing pains. People are going to not know what gets replaced or not. But we'll see how it goes. 

LR: And education and communication are really the most important tools that we have in order to most protect people's jobs, but also to inform them on how the AI can help them do their jobs. So when we look at what companies are doing out there, a lot of them are making sure that they train all of their people on the AI tools and how best to use them in order to make their job easier and make them more productive. 

LP: All right. So the workforce will evolve and adapt and learn to use LLMs. 

LR: Exactly. 

LP: All right. So going back to LAMP, any surprise findings during your LAMP journey? Maybe something you didn't expect. 

LR: I think, initially, what we found was, wow, these LLMs are really useful, which maybe we weren't so surprised about some of us weren't. However, there were some people who were very skeptical about LLMs. And it was amazing to me how we could transform that skepticism by showing them real-world examples of things that got done. 

So one of the things we did was we had this meeting where everyone took notes at the meeting. And I took copies of the notes. And I put them into our SwRI tool that we had. And I said, hey, make me a summary of these notes. Tell me what are outliers. Tell me what are common denominators. Because there were multiple tables. 

And it gave me a set of stuff that we put on the slide. And everyone looked at it and was like, wow, you compiled that? And I said, yeah, that took me five minutes, less than five minutes. Mostly, the five minutes was getting the data into the LLM. And that was an eye-opener, I think, for a lot of people. 

JE: For me, it's the acceleration is sort of unprecedented when it comes to this technology. When the LAMP program started, the big research area was prompt engineering. You could go online on GitHub. And there would be thousands of prompts, which are sort of tailor-fit verbs and terminology you want to type to the LLM in order to get a good answer, a guaranteed good answer. They call this prompt engineering, templating, so on and so forth. 

That has changed dramatically. So prompt engineering, which what the focused project started a year ago about, within that year, now we have the planning agents, which we've touched on a little bit already, where they're thinking and then acting and calling tools. Now, we have fully agentic where you type to it. And all of the planning is just kind of done behind the scenes and the tool calling and what have you. 

Even now today, just last week I was at a conference and they were talking about swarming of agents. Where now, you have one agent that is creating other agents in order to handle this task. And then, those smaller agents are telling the main agent what they did. The main agent is communicating with you. And it's only getting more and more advanced as time goes on. And we'll see where it goes. 

But the swarm agents, this is the funny statistic, they are getting so expensive in terms of token costs. They're actually rivaling the salary of junior engineers at this point. So they may be faster and more effective. But cost is starting to get a concern now. 

LR: Well, in some companies that had laid off some junior engineers actually hired junior engineers back because token costs have increased dramatically. And token costs are what you pay for. It's not exactly a word. But if you think about it in terms of a word, it's how much data you're giving to it and how much it's giving you back. 

When we first started, token costs were fairly high. New models came out. Token costs dropped. It was really pretty cheap for a while. And then, they started realizing, oh, this is actually costing us a lot in electricity and water and other elements that they weren't factoring in. And the costs went up dramatically after that. 

So now, we're seeing again, as new models that are more powerful come out, those models are very expensive. I told you we have a tool called GitHub Copilot. It's a coding tool. We just got it, like, three months ago. And it was super cheap to have at that point. A month and a half in, they said, oh, our costs are going to rise up a lot. And they gave us a usage comparison. And we're like, oh, my goodness. That's going to be crazy a lot. Then, we had to figure out how to use it more effectively, efficiently so we weren't using the most expensive models all the time. 

LP: So there are still fluctuations happening and discovery. 

LR: LLM technology is still evolving as we go on now. 

LP: All right. So LAMP has wrapped up, as you mentioned. How will LAMP discoveries be used in the future? 

LR: So some of the things we've been doing are taking things that we've learned to clients and getting funded projects from those clients, which is ultimately the goal of the internal research we do at Southwest Research Institute. Other things that we've learned have led us to be able to do things more effectively internally as well. So we're seeing an improvement in productivity in our staff, also improvement in our processes. 

So one of the things we want to do is make workflows easier for people. And those are the things that we've been able to accelerate as well. Now, from this point going forward, what we're seeing is we're still doing internal research that has LLMs as a component of it. But it's just a piece of it. So we used to see a lot of machine learning in our internal research proposals. And it would be like a piece of the proposal. Now, we're seeing that with large language models. 

Jonathan and I are working on one right now that is not part of LAMP, does use an LLM for a tiny feature of it. So it's just a piece where it says, we give it information. And it gives us more of a human-readable summary of the information. And what it might to do troubleshooting. And so that was, like, a 10% of the research project. And that's kind of how we see it going forward. It's another tool in our set of tools. 

JE: Largely, I think for a long time, AI has been, oh, you need to have some PhD, master's, very focused engineering in order to do machine learning. That time is coming to an end. The accessibility and the commoditization of AI has really hit a threshold where it has to be pitched as a component at this point. Because doing advanced research from an academic level is not having the same amount of gains as it used to. Treating it as a commodity to accelerate any kind of work across all the industries is really where I think we're going to go as a Institute. 

LP: What's your best advice for someone new to LLMs? Any tips for getting the most out of them? 

LR: So I like to use my LLM as a sounding board. When I'm doing something with the LLM, it's not like I'm trying to craft the best prompt ever. Jonathan mentioned how prompt engineering was a big thing. They were actually advertising prompt engineering jobs a year ago. And that was, like, a real job, how you crafted prompts. And that's not the case for the way I work with it. 

I do more of an interactive, conversational approach with the LLM. And I like to tell it, hey, what am I missing here? Or what do I need to add? Or what can you tell me about this data? And it's interesting because sometimes when you give the LLM data that you think you know what that data is, it will look at it and tell you something about it that you didn't see. Because you ask it to look for something you missed. And that's really important because, as people, we don't know what we don't know. Nobody knows what they don't know. And so looking for what you missed is a really powerful use for the LLM. 

LP: And, Jonathan, what's your best advice for someone new to LLMs? 

JE: I always recommend a narrow scope because if you don't understand what you are asking the LLM, the LLM is also going to struggle. I like using the example of Bob at the bar. And if you wanted Bob at the bar to get something done for you, you want to be really specific about that in order for Bob to actually help you. He's a very capable guy in everything. But if you're not clear in how you're explaining the problem to him, you might not like what he returns to you. So it's all about having a narrow scope. 

Of course, there's use-case specific because it is great as a sounding board. But even in the case where you want a general reply, be as specific as possible of what you actually want. And that might require you to do some reflective thinking on what is it that you want? It's about consistency of the tool that you're using. It's kind of if you're using a hammer, you know exactly where you want to hold the hammer, but if you held a hammer from the head and started to hit nails, you probably could still hit some nails. But it'd be a much less effective tool. 

LR: So I always liken, in my presentations, the large language model to having 10,000 interns. You can give the 10,000 interns a bunch of tasks. And they will give you stuff back. But at the end of the day, it's your job to make sure what they give you back is actually what you need and actually works. So don't treat it like it's the be all and end all and knows everything. Treat it like it's something to help you with what you're trying to do. 

LP: All right, a knowledgeable helper, but it still needs that human touch. 

LR: Yes. 

LP: All right. OK, well, thank you for telling us about LAMP and your LLM research, Lynne and Jonathan. So much helpful and useful information, as so many of us adopt LLM use at home, work, and in our daily lives. Thank you for being here. 

LR: Thank you. 

JE: Thank you for having us.

And thank you to our listeners for learning along with us today. You can hear all of our Technology Today episodes, and see photos, and complete transcripts at podcast.swri.org. Remember to share our podcast and subscribe on your favorite podcast platform.

Want to see what else we're up to? Connect with Southwest Research Institute on Facebook, Instagram, X, LinkedIn, and YouTube. Check out the Technology Today Magazine at technologytoday.swri.org. And now is a great time to become an SwRI problem solver. Visit our career page at SwRI.jobs.

Ian McKinney and Bryan Ortiz are the podcast audio engineers and editors. I am producer and host, Lisa Peña.

Thanks for listening.

[MUSIC PLAYING]