Testing the Feasibility of Positioning Using Ambient Light By Jingbin Liu, Ruizhi Chen, Yuwei Chen, Jian Tang, and Juha Hyyppä INNOVATION INSIGHTS by Richard Langley AND THEN THERE WAS LIGHT. Well, the whole electromagnetic (EM) spectrum, actually. Visible light occupies only a small portion of the spectrum, which extends from below the extremely low frequency (ELF) 3 to 30 hertz band with equivalent wavelengths of 100,000 to 10,000 kilometers through infrared, visible, and ultraviolet light and x-rays to gamma rays in the 30 to 300 exahertz band (an exahertz is 1018 hertz) with wavelengths of 10 to 1 picometers and beyond. The radio part of the spectrum extends to frequencies of about 300 gigahertz or so, but the distinction between millimeter radio waves and long infrared light waves is a little blurry. Natural processes can generate electromagnetic radiation in virtually every part of the spectrum. For example, lightning produces ELF radio waves, and the black hole at the center of our Milky Way Galaxy produces gamma rays. And various mechanical processes can be used to generate and detect EM radiation for different purposes from ELF waves for communication tests with submerged submarines to gamma rays for diagnostic imaging in nuclear medicine. Various parts of the EM spectrum have been used for navigation systems over the years. For example, the Omega system used eight powerful terrestrial beacons transmitting signals in the range of 10 to 14 kilohertz permitting global navigation on land, in the air, and at sea. At the other end of the spectrum, researchers have explored the feasibility of determining spacecraft time and position using x-rays generated by pulsars — rapidly rotating neutron stars that generate pulses of EM radiation. But the oldest navigation aids, lighthouses, used the visible part of the EM spectrum. The first lighthouses were likely constructed by the ancient Greeks sometime before the third century B.C. The famous Pharos of Alexandria dates from that era. And before the construction of lighthouses, mariners used fires built on hilltops to help them navigate. The Greeks also navigated using the light from stars, or celestial navigation.  Records go back to Homer’s Odyssey where we read “Calypso, the lovely goddess had told him to keep that constellation [the Great Bear] to port as he crossed the waters.” By around 1500 A.D., the astrolabe and the cross-staff had been developed sufficiently that they could be used to measure the altitudes of the sun or stars to determine latitude at sea. Celestial navigation was further advanced with the introduction of the quadrant and then the sextant. And determining longitude was possible by observing the moons of Jupiter (but not easily done at sea), measuring distances between the moon and other celestial bodies and, once it was developed, using a chronometer to time altitude observations. How else is light used for positioning and navigation? Early in the space age, satellites were launched with flashing beacons or with large surface areas to reflect sunlight so that they could be photographed from the ground against background stars with known positions to determine the location of the camera. We also have laser ranging to satellites and the moon and the related terrestrial LiDAR technology, as well as the total stations used by surveyors. And in this month’s column, we take a look at the simple, innovative method of light fingerprinting: the use of observations of the artificial light emitted by unmodified light fixtures as well as the natural light that passes through windows and doorways in a technique for position determination inside buildings. “Innovation” is a regular feature that discusses advances in GPS technology and its applications as well as the fundamentals of GPS positioning. The column is coordinated by Richard Langley of the Department of Geodesy and Geomatics Engineering, University of New Brunswick. He welcomes comments and topic ideas. Over the years, various localization technologies have been used to determine locations of people and devices in an absolute or relative sense. Relative positioning methods determine a location relative to another one in a local coordinate framework, while absolute positioning techniques fix an absolute location in a specific coordinate framework. In the past, people observed the positions (orientation angles) of a celestial body (such as the sun, the moon, or a star) to determine their locations on the Earth, which is known as celestial navigation (see FIGURE 1). The locations are resolved by relating a measured angle between the celestial body and the visible horizon to the Nautical Almanac, which is a knowledge base containing the coordinates of navigational celestial bodies and other relevant data. Other than an observation device, celestial navigation does not rely on any infrastructure, and hence it can be used virtually anywhere on the globe at anytime, weather permitting. Nowadays, an increasing number of applications, location-based services, and ambient intelligence largely require positioning functions across various environments due to increasing mobility of people and devices. In particular, the development of robotics for a number of purposes requires the support of localization capability in various conditions where positioning infrastructure may be missing. Various positioning technologies share an intrinsic characteristic that a positioning solution is resolved by using the dependency between spatial locations and a set of physical observables. The dependency may be expressed in the form of either a deterministic function model or a probabilistic model. A deterministic model expresses the dependency between locations and observables in a closed-form function, while a probabilistic model defines the dependency between locations and observables in the Bayesian sense. Depending on the form of dependency, different mathematical models have been used for position resolution.   For example, satellite-based GNSS positioning derives the location of a user’s receiver based on radio frequency (RF) signals transmitted by the satellite systems. GNSS positioning is grounded in accurate time determination: the time differences between the transmitted and the received radio signals denote signal travel times (observables), which are then converted into distance measurements between the satellite and the user antenna. Using the distance measurements between the user antenna and four different satellites, the receiver can obtain three-dimensional receiver coordinates in a global reference frame and the time difference between the receiver and satellite clocks. The dependency between user location and a set of distance observables can be expressed in a simplified equation: (1) where ρi is an observed range between the ith satellite and the receiver, (x,y,z)i is the position of the ith satellite, (x,y,z) is the position of the receiver to be estimated, γ denotes errors in the range observable, δt and c are receiver clock error and the speed of  light, respectively (the sign of the clock term is arbitrary, but must be used consistently). It is obvious that GNSS positioning relies strongly on the visibility of the GNSS constellation — the space infrastructure — as it requires line-of-sight visibility of four or more satellites. The positioning capability is degraded or totally unavailable in signal-blocked environments, such as indoors and in urban canyons.  An example of Bayesian positioning is to use various signals of opportunity (SOOP) — signals not originally intended for positioning and navigation. They include RF signals, such as those of cellular telephone networks, digital television, frequency modulation broadcasting, wireless local area networks, and Bluetooth, as well as naturally occurring signals such as the Earth’s magnetic field and the polarized light from the sun. Indicators of these signals, such as signal strengths and signal quality, are dependent on locations in the Bayesian sense. The dependency between signal indicators and locations is expressed in a probabilistic model:   (2) where  signifies a dependency between a set of physical signals and locations, I denotes indicators of SOOP signals, L denotes location, and P(i|l) is the probability that signal indicators (i) are observed at location (l). Positioning resolution involves finding a location that yields the maximum a posteriori probability given a specific set of observables. Bayes’ Rule for computing conditional probabilities is applicable in the positioning estimation, and a family of Bayesian inference methods has been developed (see Further Reading).  An inertial navigation system (INS) is a typical relative positioning technology, and it provides the estimation of moved distance, direction, and/or direction change. A commonly used INS consists of accelerometers, gyroscopes, and a compass. It is self-contained and needs no infrastructure in principle to operate. However, the sensors yield accumulated positioning errors, and they need extra information for calibration. For example, in a GNSS/INS combined system, the INS needs to be calibrated using GNSS positioning results. To achieve an enhanced positioning performance in terms of availability, accuracy, and reliability, different positioning technologies are commonly integrated to overcome the limitations of individual technologies in applicability and performance. This article discusses the feasibility of ambient light (ambilight) positioning, and we believe it is the first time that ambilight has been proposed as a positioning signal source. We propose the use of two types of observables of ambient light, and correspondingly two different positioning principles are applied in the positioning resolution. Our solution does not require any modifications to commonly used sources of illumination, and it is therefore different from other indoor lighting positioning systems that have been proposed, which use a modulated lighting source. Ambilight positioning does not require extra infrastructure because illumination infrastructure, including lamps and their power supply and windows, are always necessary for our normal functioning within spaces. Ambilight exists anywhere (indoor and outdoor), anytime, if we consider darkness as a special status of ambient light. Ambilight sensors have been sufficiently miniaturized and are commonly used. For example, an ambilight sensor is used in a modern smartphone to detect the light brightness of the environment and to adaptively adjust the backlight, which improves the user vision experience and conserves power. Additionally, ambilight sensors are also widely used in automotive systems to detect the light intensity of environments for safety reasons. Therefore, ambilight positioning can use existing sensors in mobile platforms. This article presents the possibilities and methods of ambilight positioning to resolve both absolute and relative positioning solutions, and which can be integrated as a component in a hybrid positioning system.  Absolute Positioning Using Ambilight Spectral Measurements  The essence of localization problems is to resolve the intrinsic dependency of location on a set of physical observables. Therefore, a straightforward idea is that the type of observables applicable to positioning can be determined once the location-observables dependency is established. The feasibility is validated when the location-observables dependency is confirmed in the sense of necessary and sufficient conditions. Ambient light is a synthesis of artificial light sources and natural light. The light spectrum is defined by the distribution of lighting intensity over a particular wavelength range. Researchers have reported development of sensor technology that has a spectral response from 300 to 1450 nanometers (from ultraviolet through infrared light). The spectrum of ambient light is mainly determined by colors of reflective surfaces in the circumstance, in addition to that of artificial and natural light sources. Therefore, intensity spectrum measurements are strongly correlated with surrounding environments of different locations. The traditional fingerprinting method can be used to resolve the positioning solution.  The fingerprinting approach makes use of the physical dependency between observables and geo-locations to infer positions where signals are observed. This approach requires the knowledge of observable-location dependency, which comprises a knowledge database. The fingerprinting approach resolves the most likely position estimate by correlating observed SOOP measurements with the knowledge database. The related fingerprinting algorithms include K-nearest neighbors, maximum likelihood estimation, probabilistic inference, and pattern-recognition techniques. These algorithms commonly consider moving positions as a series of isolated points, and they are therefore related to the single-point positioning approach. In addition, a “hidden Markov” model method has been developed to fuse SOOP measurements and microelectromechanical systems (MEMS) sensors-derived motion-dynamics information to improve positioning accuracy and robustness. In the case of ambilight positioning, prior knowledge is related to structure layout information, including the layout of a specific space, spatial distribution of lighting sources (lamps), types of lighting sources, and windows and doors where natural light can come through. Spatial distribution of lighting sources is normally set up together with power supplies when the structure is constructed, and their layout and locations are not usually changed thereafter. For example, illumination lamps are usually installed on a ceiling or a wall in fixed positions, and the locations of doors and windows, through which light comes, are also typically fixed throughout the life of a building. Therefore, the knowledge database of lighting conditions can be built up and maintained easily through the whole life cycle of a structure. In practice, a specific working region is divided into discrete grids, and intensity spectrum measurements are collected at grid points to construct a knowledge database. The grid size is determined based on the required spatial resolution and spatial correlation of spectrum measurements. The spatial correlation defines the degree of cross-correlation of two sets of spectrum measurements observed at two separated locations. We measured the spectrum of ambient light with a two-meter grid size in our library. The measurements were conducted using a handheld spectrometer. FIGURE 2 shows a set of samples of ambilight spectrum measurements, and the corresponding photos show the circumstances under which each spectrum plot was collected. These spectral measurements show strong geo-location dependency. Spectrum differences of different locations are sufficiently identifiable. TABLE 1 shows the cross-correlation coefficients of spectral measurements of different locations. The auto-correlation coefficients of spectral measurements of a specific location are very close to the theoretical peak value of unity, and the cross-correlation coefficients of spectra at different locations are significantly low. Therefore, the correlation coefficient is an efficient measure to match a spectrum observable with a geo-referred database of ambilight spectra. FIGURE 2. Ambilight spectral measurements of nine locations in the library of the Finnish Geodetic Institute (arbitrary units). The photos below the spectrum plots show the circumstances under which the corresponding spectral measurements were collected. TABLE 1. Correlation coefficient matrix of spectral measurements of different locations. Relative Positioning Using Ambilight Intensity Measurements Total ambilight intensity is an integrated measure of the light spectrum, and it represents the total irradiance of ambient light. In general, a lamp produces a certain amount of light, measured in lumens. This light falls on surfaces with a density that is measured in foot-candles or lux. A person looking at the scene sees different areas of his or her visual field in terms of levels of brightness, or luminance, measured in candelas per square meter. The ambilight intensity can be measured by a light detector resistor (LDR), and it is the output of an onboard 10-bit analog-to-digital converter (ADC) on an iRobot platform, which is the platform for a low-cost home-cleaning robot as shown in FIGURE 3. FIGURE 3. The iRobot-based multi-sensor positioning platform, which is equipped with a light sensor and other versatile positioning sensors as marked in the figure. We designed a simple current-to-voltage circuit based on an LDR and a 10-kilohm resistor, and the integrated analog voltage is input into the iRobot’s ADC with a 25-pin D-type socket, which is called the Cargo Bay Connector. FIGURES 4 and 6 show that the LDR sensor was not saturated during the test whenever we turned the corridor lamps on or off. Since the output of the light sensor was not calibrated with any standard light source, the raw ADC output rather than real values of physical light intensity was used in this study. During the test, the iRobot platform ran at a roughly constant speed of 25 centimeters per second, and the response time of the LDR was 50 milliseconds according to the sensor datasheet. The sampling rate of light intensity measurements was 5 Hz. Thus, the ADC could digitalize the input voltage in a timely fashion. FIGURE 4. Total irradiance intensity measurements of ambient light in a closed space. The estimated lamp positions (magenta points) can be compared to the true lamp positions (green points). FIGURE 6. Total irradiance intensity measurements of ambient light in the open corridor of the third floor. We conducted the experiments with the iRobot platform in two corridors in the Finnish Geodetic Institute building. The robot was controlled to move along the corridors, and it collected measurements as it traveled. The two corridors represent two types of environment. The corridor of the first floor is a closed space where there is no natural light, and the corridor of the third floor has both natural light and artificial illuminating light. The illuminating fluorescent lamps are installed in the ceiling. In a specific environment, fluorescent lamps are usually installed at fixed locations, and their locations are not normally changed after installation. Therefore, the knowledge of lamp locations can be used for positioning. Ambilight positioning is relatively simple in the first case where there is no natural light in the environment and all ambilight intensity comes from artificial light. Because the fluorescent lamps are separated by certain distances, the intensity measurements have a sine-like pattern with respect to the horizontal distance along the corridor. The sine-like pattern is a key indicator to be used for detecting the proximity of a lamp. As shown in Figures 4 and 6, raw measurements of ambilight intensity and smoothed intensity have a sine-like pattern. Because raw intensity measurements have low noise, either raw measurements or smoothed intensity can be used to detect the proximity of a lamp. Figure 4 also shows the results of detection and the comparison to the true lamp positions. There are four fluorescent lamps in this corridor test. The first three were detected successfully, and the estimated positions are close to true positions with a root-mean-square (RMS) error of 0.23 meters. The fourth lamp could not be detected because its light is blocked by a shelf placed in the corridor just below the lamp as shown in FIGURE 5. Figure 4 shows the sine-like intensity pattern of the fourth lamp did not occur due to the blockage. FIGURE 5. The light of the fourth lamp in the corridor is blocked by shelves, and the corresponding sine-like light pattern does not appear. On the third floor, the situation is more complicated because there is both natural light and incandescent lamps in the corridor. Natural light may come in from windows, which are located at multiple locations on the floor. In addition, the light spectrum in the corridor may be interfered with by light from office rooms around the floor. To recover the sine-like intensity pattern of the lamps, the intensity of the background light was measured when the incandescent lamps were turned off. Therefore, the calibrated intensity measurements of illuminating lamps can be calculated as follows:   (3) where Ia is the intensity measurements of composite ambient light, Ib is the intensity measurements of background light, and Ic is the intensity measurements of the calibrated ambient light of the illuminating lamps. Figure 6 shows the intensity measurements of composite ambient light, background light, and calibrated lamp light. In addition, the intensity measurements of calibrated lamp light are smoothed by an adaptive low-pass filter to mitigate noise and interference. The intensity measurements of smoothed lamp light were used to estimate the positions of the lamps according to the sine-like pattern. The estimated lamp positions were compared to the true lamp positions, and the errors are shown in FIGURE 7. The estimated lamp positions have a mean error of 0.03 meters and an RMS error of 0.79 meters. In addition, for the total of 15 lamps in the corridor, only one lamp failed to be detected (omission error rate = 1/15) and one lamp was detected twice (commission error rate = 1/15).  Discussion and Conclusion Ambilight positioning needs no particular infrastructure, and therefore it does not have the problem of infrastructure availability, which many other positioning technologies have, limiting their applicability. For example, indoor positioning systems using Wi-Fi or Bluetooth could not work in emergency cases when the power supply of these devices is cut off. What ambilight positioning needs is just the knowledge of indoor structure and ambilight observables. The lighting conditions of an indoor structure can be reconstructed based on the knowledge of the layout structure whenever illuminating lamps are on or off. Thus, ambilight observables can be related to the layout structure to resolve positioning estimates as we showed in this article.  Besides indoor environments, the methods we have presented are also applicable in many other GNSS-denied environments, such as underground spaces and long tunnels. For example, the Channel Tunnel between England and France has a length of 50.5 kilometers, and position determination is still needed in this kind of environment. In such environments, there is usually no natural light, and the intensity of illuminating lamps has a clear sine-like pattern. In particular, ambient light positioning is promising for robot applications when a robot is operated for tasks in a dangerous environment where there is no infrastructure for other technical systems such as Wi-Fi networks. Given the knowledge of the lighting infrastructure acquired from the construction layout design, the method of ambilight positioning can be used for robot localization and navigation. Our tests have shown also that the proposed ambilight positioning methods work well with both fluorescent lamps and incandescent lamps, as long as the light intensity sensor is not saturated.  A clear advantage of the technique is that the illuminating infrastructure and the structure layout of these environments are kept mostly unchanged during their life cycle, and the lighting knowledge can be constructed from the structure design. Hence, it is easy to acquire and maintain these knowledge bases. The hardware of ambient light sensors is low-cost and miniature in size, and the sensors can be easily integrated with other sensors and systems. Although a spectrometer sensor is not currently able to be equipped with a mobile-phone device, the proposed ambilight positioning techniques can still be implemented with a modern mobile phone in several ways. For example, an economical way would be to form a multispectral camera using a selection of optical filters of selected bands or a miniature adjustable gradual optical filter. The spectral resolution then is defined by the bandwidth of the band-pass optical filters and the optical characteristics of the gradual optical filter. Other sensors, such as an acousto-optic tunable filter spectrometer and a MEMS-based Fabry-Pérot spectrometer, could also be used to measure the spectrum of ambilight in the near future. With such techniques, ambilight spectral measurements can be observed in an automated way and with higher temporal resolution.  Acknowledgments The work described in this article was supported, in part, by the Finnish Centre of Excellence in Laser Scanning Research (CoE-LaSR), which is designated by the Academy of Finland as project 272195. This article is based on the authors’ paper “The Uses of Ambient Light for Ubiquitous Positioning” presented at PLANS 2014, the Institute of Electrical and Electronics Engineers / Institute of Navigation Position, Location and Navigation Symposium held in Monterey, California, May 5–8, 2014. JINGBIN LIU is a senior fellow in the Department of Remote Sensing and Photogrammetry of the Finnish Geodetic Institute (FGI) in Helsinki. He is also a staff member of the Centre of Excellence in Laser Scanning Research of the Academy of Finland. Liu received his bachelor’s (2001), master’s (2004), and doctoral (2008) degrees in geodesy from Wuhan University, China. Liu has investigated positioning and geo-reference science and technology for more than ten years in both industrial and academic organizations.  RUIZHI CHEN holds an endowed chair and is a professor at the Conrad Blucher Institute for Surveying and Science, Texas A&M University in Corpus Christie. He was awarded a Ph.D. degree in geophysics, an M.Sc. degree in computer science, and a B.Sc. degree in surveying engineering. His research results, in the area of 3D smartphone navigation and location-based services, have been published twice as cover stories in GPS World. He was formerly an FGI staff member. YUWEI CHEN is a research manager in the Department of Remote Sensing and Photogrammetry at FGI. His research interests include laser scanning, ubiquitous LiDAR mapping, hyperspectral LiDAR, seamless indoor/outdoor positioning, intelligent location algorithms for fusing multiple/emerging sensors, and satellite navigation. JIAN TANG is an assistant professor at the GNSS Research Center, Wuhan University, China, and also a senior research scientist at FGI. He received his Ph.D. degree in remote sensing from Wuhan University in 2008 and focuses his research interests on indoor positioning and mapping. JUHA HYYPPA is a professor and the head of the Department of Remote Sensing and Photogrammetry at FGI and also the director of the Centre of Excellence in Laser Scanning Research. His research is focused on laser scanning systems, their performance, and new applications, especially those related to mobile laser scanning and point-cloud processing. FURTHER READING • Authors’ Conference Paper “The Uses of Ambient Light for Ubiquitous Positioning” by J. Liu, Y. Chen, A. Jaakkola, T. Hakala, J. Hyyppä, L. Chen, R. Chen, J. Tang, and H. Hyyppä in Proceedings of PLANS 2014, the Institute of Electrical and Electronics Engineers / Institute of Navigation Position, Location and Navigation Symposium, Monterey, California, May 5–8, 2014, pp. 102–108, doi: 10.1109/PLANS.2014. 6851363. • Light Sensor Technology “High-Detectivity Polymer Photodetectors with Spectral Response from 300 nm to 1450 nm” by X. Gong, M. Tong, Y. Xia, W. Cai, J.S. Moon, Y. Cao, G. Yu, C.-L. Shieh, B. Nilsson, and A.J. Heeger in Science, Vol. 325, No. 5948, September 25, 2009, pp. 1665–1667, doi: 10.1126/science.1176706. • Light Measurement “Light Intensity Measurement” by T. Kranjc in Proceedings of SPIE—The International Society for Optical Engineering (formerly Society of Photo-Optical Instrumentation Engineers), Vol. 6307, Unconventional Imaging II, 63070Q, September 7, 2006, doi:10.1117/12.681721. • Modulated Light Positioning “Towards a Practical Indoor Lighting Positioning System” by A. Arafa, R. Klukas, J.F. Holzman, and X. Jin in Proceedings of ION GNSS 2012, the 25th International Technical Meeting of the Satellite Division of The Institute of Navigation, Nashville, Tennessee, September 17–21, 2012, pp. 2450–2453. • Application of Hidden Markov Model Method “iParking: An Intelligent Indoor Location-Based Smartphone Parking Service” by J. Liu, R. Chen, Y. Chen, L. Pei, and L. Chen in Sensors, Vol. 12, No. 11, 2012, pp. 14612-14629, doi: 10.3390/s121114612. • Application of Bayesian Inference “A Hybrid Smartphone Indoor Positioning Solution for Mobile LBS” by J. Liu, R. Chen, L. Pei, R. Guinness, and H. Kuusniemi in Sensors, Vol. 12, No. 12, pp. 17208–17233, 2012, doi:10.3390/s121217208. • Ubiquitous Positioning “Getting Closer to Everywhere: Accurately Tracking Smartphones Indoors” by R. Faragher and R. Harle in GPS World, Vol. 24, No. 10, October 2013, pp. 43–49. “Hybrid Positioning with Smartphones” by J. Liu in Ubiquitous Positioning and Mobile Location-Based Services in Smart Phones, edited by R. Chen, published by IGI Global, Hershey, Pennsylvania, 2012, pp. 159–194. “Non-GPS Navigation for Security Personnel and First Responders” by L. Ojeda and J. Borenstein in Journal of Navigation, Vol. 60, No. 3, September 2007, pp. 391–407, doi: 10.1017/S0373463307004286.