Built a Bot. Charlotte Wagner at SEI in Boston was the one who technically developed SEI’s bot using AI. Tools of this kind can truly be useful, according to her. – Analyses that previously required a team of researchers working for a year can now be produced very quickly, she says.
Hi there Charlotte Wagner, senior scientist at SEI. Was it complicated to develop the prompts for the bot?
– No, not really. But because large language models (LLMs) are non-deterministic and interpret questions and prompts slightly differently each time, careful, iterative adjustment of the prompts is required. We had to do quite a bit of experimentation.
How long did it take?
– We built a first working prototype in just a couple of days. After that, a significant amount of work was needed to refine the prompts. A large part of the work then involved compiling and curating the scientific literature. We had to review several hundred studies on the effects of offshore development on the marine environment and validate that the bot reproduced our review findings accurately. In total, the process took about six months.
Are you satisfied with the result?
– I think it is an excellent example of how LLMs can support scientifically informed decision-making and ultimately generate better environmental policy. It is also exciting to see how far LLMs have come in recent years. Analyses that previously required a team of researchers working for a year can now be produced very quickly. The results are also often more useful when it is possible to start from specific interests and questions rather than relying on a static literature summary. The insights become more relevant.
Any plans to develop it further?
– Yes, absolutely. We are currently seeking funding for that. We would like to improve it technically, expand the knowledge base and address some of its limitations. We are also exploring new possibilities, such as generating maps with it.
Is there any risk that confidential material could leak?
– As long as users only ask questions about the bot’s knowledge base, there is no major cause for concern. But as with all interactions on commercial LLM platforms, we recommend that users avoid handling confidential material. In the future, however, we would like to develop versions that provide full control.