Passive screen-camera communication

More Info


The advancement of wireless communication technologies has transformed how we exchange information in our daily lives. However, the increasing demand for wireless communication faces challenges due to limited radio wave bandwidth availability. In this context, visible light communication has emerged as a promising alternative, utilizing visible light for data transmission and reception. Screen-based visible light communication, also known as screen-camera communication, is a highly appealing option due to its exceptional user accessibility and seamless integration with existing infrastructure. It enables data to be embedded imperceptibly for humans within video content, ensuring a seamless and unobtrusive transmission.

Screens can be classified into two types: active screens, which emit their own light, and passive screens which harness ambient light as a more energy-efficient light source. Existing research has largely explored the use of active screens in screen-camera communication. This work focuses on the feasibility and implementation of a passive screen-camera communication link, addressing both data embedding within video scenes and decoding using an Android smartphone.

To accomplish this objective, an Android application is developed to decode data transmitted through a passive screen. Unlike most previous approaches that perform decoding in a non-real-time manner, this work implements a real-time application. However, because of the constraints imposed by the capturing rates of real-time cameras, both the transmission and reception are limited to low frame rates, specifically 15 frames per second transmission rate. It is anticipated that advancements in technology will eventually overcome this limitation. However, currently, the low transmission rate exhibits noticeable flickering. To reduce this issue, two novel encoding techniques were introduced, slightly reducing the flicker.

Through the exploration of a passive screen-camera communication link, this research further contributes to the advancement of visible light communication technologies. The proposed decoding techniques eliminate the reliance on visual screen reference points and address the challenge of real-time hand motion filtering, enhancing the practicality and usability of passive screen-camera communication systems in real-world scenarios. Finally, a proof-of-concept Android application was developed, effectively synchronizing audio playing on the smartphone with the video content displayed on a passive screen.

The response time performance of the passive screen-camera link was measured to be 530 ms when using a static phone holder and 1071 ms when handholding the phone.