Software Tool to Assess User Attentiveness During Online Engagements

This technology offer is a software tool that can be used to assess the level of attentiveness of a user during online engagements, such as during an online meeting, or an online lesson. The software tool is run in the background when a user launches an online meeting platform.

The software tool consists of a server software and a client software, and realises three functions, namely, user attentiveness measurement, background audio noise estimation, and automated video editing with video segment output. Multiple client software can be used with a single server software at the same time. The server software handles incoming data from the clients and displays their status on a web-based dashboard. After a meeting ends, the client software will automatically output a video clip showing segments where the user has the highest level of attentiveness. This video clip can be used for further downstream analytics by the user or by the meeting/lesson organiser.

The software tool can be used together with online meeting platforms. The prototype source code is available for demonstration purposes.
 

This technology offer is capable of detecting a user’s face orientation and distance from the computer screen (camera) to assess the level of attentiveness. It can also detect the user’s voice pitch variations and the background audio noise level to determine when the user is most attentive. The built-in automatic video editing function can utilise these video and audio cues and output a condensed video clip to show all the segments of high user attentiveness.

Software Feature Specifications
The key feature specifications are described below:
•    The software supports multiple users at any one time
•    An authentication feature allows the user to configure IP address and port number
•    A login feature allows the user to register ID name
•    A display feature allows the user to check the real-time status
•    A video storage feature allows the user to review the previous status

Software Performance Specifications
The key performance specifications are described below:
•    The software can be used together with Zoom/Microsoft Teams software concurrently
•    The software supports multiple displays
•    The software is capable of loading in 5 seconds
•    The software can process each video frame within 0.5 seconds
•    The software can update data with a latency of no greater than 1 minute
 

This technology offer can be implemented and used in various areas, such as for:
•    Online education/lessons
•    Web conferencing
•    Online retail
•    Online recruitment
•    Artificial Intelligence (AI) concierge services
•    Robotics

Nowadays, more and more companies and organizations support telecommuting. Schools have also implemented online courses for students. There are many popular web conferencing applications (e.g., Zoom, Microsoft Teams) and online course platforms. However, when attending online meetings or online lessons, people may not focus well; this may be caused by environmental or other factors. This issue needs to be solved, as there is an increasing demand for online meetings and lessons. A simple and light software which can monitor the attention status of online engagements is necessary for remote working or remote education. Furthermore, social media platforms today have diversified methods of posting information. The volume of content to be distributed through social networking services (SNS) and video hosting services has been rapidly increasing. Automation of content editing/analysis/checking is required in order to improve quality of service.

This technology offer enables the online event organiser e.g., online lesson provider, meeting organiser, etc., to assess the level of attentiveness of the event attendees. With this knowledge, the online event organiser could fine tune its online engagement methods and presentation material so that future online engagements can reap users with high attentiveness. This technology also enables SNS platforms or video hosting services to develop automated content editing/analysis/checking functions for video clips uploaded by users.

The technology owner is keen to out-license this patented technology to software development companies involved in making software for web meeting platforms, online content platform providers, or online service providers e.g., social networking service (SNS) providers.