This manual explains the procedures from the introduction to the operation of the AI automated conversation function.
*This feature is available to customers who have subscribed to an optional service.
If you wish to use this feature, please contact us.
Compatible models: ugo Pro (G4), ugo OS: v4.5.0 or later
◆ Table of Contents
• Key improvements to the AI auto-conversation function
• Tools
◆ Function Overview
The AI-powered auto-conversation feature enhances both on-site usability and conversation quality by improving ugo's response speed and integrating an AI agent. This feature enables ugo to deliver advanced customer service by combining dialogue with visual information through digital signage.
◆ Basic Operation
The traditional concept of "modes" has been eliminated; operation is now controlled simply by toggling the "auto-conversation icon" on or off from the operation screen. Auto-conversation can also be switched on or off via commands. Flow and auto-conversation are controlled and executed independently.
■Auto-conversation ON
Run a conversation with Flow in parallel:
When no people are detected, the system displays greetings and signage based on the Flow. When a person is detected within the configured detection range and a conversation begins, the running Flow is paused (interrupted) and automatically resumes once the conversation ends. During the conversation, the AI agent draws on Q&A data, web search tools, weather forecast tools, and transit information tools to answer questions.
■Auto-conversation OFF
Standby status:
In this state, ugo only executes Flow and does not engage in auto-conversation. Running Flows are unaffected and continue to operate as usual.
◆ Key improvements to the AI auto-conversation function
Improved response speed:
The time between the customer finishing speaking and ugo beginning to respond is shorter than with conventional auto-guidance features, enabling more natural conversation.
*A stable network connection is required.
AI Agent:
The AI automatically selects the appropriate tool (knowledge search, weather forecast, transit information, or web search) based on the conversation content, improving both the efficiency and accuracy of answer generation.
Digital signage integration function:
Information is visually reinforced by displaying conversation subtitles, along with images and videos attached to Q&A, on the digital signage.
Digital signage speaker function:
When ugo engages in conversation using the AI auto-conversation feature, the spoken audio is output from the speaker at the bottom of the signage. This makes it easier for the questioner to understand what ugo is saying.
English support (automatic translation):
It can automatically detect the language of the recognized speech and translate the Q&A data content in real time to generate a response. Even if your registered Q&A data is only in Japanese, the system can automatically translate and respond when spoken to in English.
*There is no feature for translating the content displayed on the signage.
◆ Tools
Thanks to the tools and features available to its AI agent, ugo can handle a wide range of tasks, from on-site guidance to real-time information delivery. The AI analyzes each question and provides an answer using the appropriate tools.
Knowledge search tool:
Provides specific on-site information not available on the internet, based on pre-registered Q&A data (on-site information). (For example, details such as the location of restrooms within the facility and event information.)
Web search function tool:
Performs a web search using your registered domains and incorporates the results into the response. Up to 20 domains can be registered.
Weather forecasting tools:
If you ask about the weather, a weather forecast tool is automatically called and ugo provides the answer. In the agent settings, under "Additional Instructions" (prompt), define ugo's current location (prefecture or city/town name). Alternatively, you can specify the location directly in your question.
The signage displays weather forecasts for up to three days, including the current day.
Train Transfer information tool (Japan Only):
- ugo provides train transfer information (routes) from your departure point to your destination, with the route details and fare displayed on the signage.
If the questioner does not specify a starting point, the search is performed based on ugo's current location (the nearest station name).
This can also be configured in the "Additional Instructions" (prompt) section within the agent settings. - The signage displays a QR code linking to an API site that provides transit information. Customers can scan the QR code with their smartphone or another device to view detailed information.
*This service does not support limited express fares such as the Shinkansen.
*This service only provides transfer information for railways within Japan.
LLM response function:
Uses the LLM model's internal knowledge to generate flexible dialogue.
It can respond naturally to casual conversation and simple questions from customers (e.g., "Who designed the Tokyo Station building?").
*The LLM model does not have the ability to learn from responses or conversation history.
◆ Hardware Configuration
Please check the names of the various parts of the ugo Pro with the signage installed, as well as the included interfaces.
| Component | Role |
| Display Base Unit | Attaches to the top pillar of the ugo Pro lifter unit. It consists of the components for mounting the signage to the lifter unit, as well as the computer that controls the signage. |
| Digital Signage |
Displays images and videos according to instructions from the "Signage Image Display" command. Images and videos can be displayed via commands, Flow, and WorkPlan. With the AI auto-conversation feature, the signage displays subtitles of both the questioner's speech (as recognized by ugo) and ugo's responses. When an answer is generated using the knowledge search tool, the image linked to the relevant Q&A is also displayed. [Display Size]
[Supported File Formats]
*If the file size differs, it will be scaled to fit the signage. |
| Speaker | When ugo speaks using the AI auto-conversation feature, its voice is output from the speaker located at the bottom of the signage. |
| Microphone | Used for voice input with the AI auto-conversation feature. When a conversation begins, the system automatically switches to voice input from this microphone, which is located next to the face display. |
| Cable Reel | Stores the cable that connects the microphone to the Display Base Unit. It attaches to the lifter unit via a magnet. |
🚨 Essential requirements for stable operation
A stable network connection is essential for the AI auto-conversation feature to operate reliably. The AI auto-conversation feature cannot be used while ugo is offline.
- Network connection: ugo requires an active network connection.
- Stable bandwidth: A stable bandwidth of at least 5 Mbps download and 1 Mbps upload must be secured.
◆ 8 Setup Steps
To understand how the AI auto-conversation feature works and to get started with stable operation, please follow the steps below in order.
STEP 1: Install the signage and microphone unit
First, install and configure the signage and microphones.
▶︎ [ Click here for details on STEP 1 ]
STEP 2: Register knowledge and Q&A
Next, create the question-and-answer data (Q&A data) that you want ugo to respond with, and register it in the knowledge base. Prepare specific answers to anticipated customer questions, such as facility information and frequently asked questions.
This data has a significant impact on the quality of ugo's conversations.
▶︎ [ Click here for details on STEP 2 ]
STEP 3: Register resources and attach images to the Q&A.
Register the images/videos you want to display on the signage and link them to the Q&A data.
▶︎ [ Click here for details on STEP 3 ]
STEP 4: AI Agent Setup
You can define the AI's role and tone of voice through "Additional Instructions" and configure which tools it can use.
By setting "Additional Instructions" (prompts), you can exercise more granular control over ugo's responses.
▶︎ [ Click here for details on STEP 4 ]
STEP 5: Set up auto-conversation
- Configure the method and range of human detection, as well as the greetings to use at the start and end of conversations.
▶︎ [ Click here for details on STEP 5 ]
STEP 6: Test the conversation
Test the response on the ugo Portal or on a real device to improve the quality of the AI's response.
▶︎ [ Click here for details on STEP 6 ]
STEP 7: Operate along with Flow
Use Flow to activate the AI auto-conversation feature and begin operation, including digital signage display.
▶︎ [ Click here for details on STEP 7 ]
STEP 8: Improve Q&A and additional instructions using conversation history
Analyze conversation history to continuously expand Q&A data and update "Additional Instructions," improving ugo's auto-conversation over time.
▶︎ [ Click here for details on STEP 8 ]
◆ Improvement Cycle
ugo can be made smarter every day by following this improvement cycle: analyze customer conversation history, then add or modify Q&A data and "Additional Instructions."