U.S. President Donald Trump announced on Tuesday that the White House will launch America.gov, a AI chatbot designed to help people find government services and other information.

Google has confirmed that it is a partner in this release, with its AI model Gemini participating in it. However, it is not yet clear whether any other AI companies have also made contributions.

Trump posted on X, saying: "Rather than forcing citizens to search through an endless labyrinth of thousands of government websites and rules... you will now have a single entry point that can answer every question."

Making government services more easily findable and usable is a goal worth pursuing, but is the AI chatbot really the best way to achieve this goal? Large language models are not flawless and can still be prone to hallucinations. In fact, CNN recently reported that the US military nearly launched a military action against a Chinese ship because they believed the ship was carrying nuclear weapon components. It was only at the last moment that the military canceled the operation, as they realized that the so-called threat was just a AI hallucination.

If someone receives incorrect information from America.gov, it probably won't trigger a nuclear war, but the errors could still have serious consequences. People may rely on this chatbot for information regarding applying for food subsidies, renewing visas, or filing taxes. However, a mistake could mean missing a deadline, having their benefits denied, or facing fines.