Research Article | Open Access
Yuan Tong, Mike Pivnenko, Daping Chu, "Implementation of 10-Bit Phase Modulation for Phase-Only LCOS Devices Using Deep Learning", Advanced Devices & Instrumentation, vol. 2020, Article ID 9515747, 10 pages, 2020. https://doi.org/10.34133/2020/9515747
Implementation of 10-Bit Phase Modulation for Phase-Only LCOS Devices Using Deep Learning
A deep learning model was built to optimize the phase flicker performance for given pulse width modulation (PWM) driving patterns of a liquid crystal on silicon (LCOS) device. 10-bit phase modulation was physically realized with a phase flicker of 0.055% over 1024 addressed phase levels in respect to the total modulation range of and a separation probability of 62.63% for the phase to stay within its level without overlapping with the adjacent ones. The spatial information bandwidth of the full high-definition (HD) LCOS device at 100 Hz was improved by 25%, from ~1.6 Gb/sec to ~2 Gb/sec.
Spatial light modulators (SLMs) are widely used in many integrated photonics and electronic systems for a wide range of applications, from conventional and holographic displays [1–3] to telecommunication wavelength-selective switches (WSS) [4, 5] and biomedical optical tweezers [6, 7].
Currently, there are two main types of SLMs, the MEMS-based digital micromirror devices (DMDs) and the liquid crystal-based ones such as phase-only liquid crystal on silicon (LCOS) devices. The former is fast switching in the order of 101 μs but can only modulate the amplitude of light in a binary fashion; hence, it has poor optical efficiency. The latter switches slower than DMDs, in the order of 101 ms, but can modulate light in a phase-only manner with several hundred phase levels, and hence, it has high efficiency. Except for the optical efficiency, another key performance property of the SLMs is their spatial information bandwidth, i.e., the product of the display resolution, refresh rate, and number of modulation levels. Compared to DMDs, LCOS devices have significantly lower spatial information bandwidths. A full high-definition (HD) LCOS device has a bandwidth of ~1.6 Gb/sec (), while a DMD with the same resolution can provide a bandwidth of ~50 Gb/sec (). It would be hugely beneficial if LCOS devices with an unprecedented level of spatial information bandwidth could be realized through the fundamental research.
One way to enlarge the information bandwidth is to increase the number of phase modulation levels, i.e., phase resolution, of the LCOS devices. In digitally driven LCOS devices, phase resolution can be degraded by the temporal fluctuation of the phase response, i.e., phase flicker. This instability of the phases comes from the competition between the change of the electrical driving force and LC molecule relaxation, resulting in a fast-changing phase shift error . A large phase flicker can lead to significant overlapping between the adjacent phase levels and consequently affects the accuracy and resolution of the phase modulation.
Yang and Chu  have reviewed the existing methods for phase flicker minimization, including high-frequency field inversion, DC balancing, temperature tuning, LC cell thickness, and pulse width modulation (PWM) optimization. The authors’ previous work  has successfully reduced the phase flicker to 0.09% over 256 addressed phase levels in respect to the total phase modulation range of by optimizing the PWM driving waveforms. However, the optimization process was time-consuming because of manually selecting the suitable driving patterns from billions of combinations. In addition, the actual pattern selection, i.e., the ways of splitting the selected long pulses and distributing them in a uniform manner, is only proven to work for a certain LC mixture used in a particular device, and it is not guaranteed to have the same effect for other LC mixtures or other LCOS devices.
In order to address these issues, we propose a new approach for the driving pattern selection, i.e., to predict and optimize the phase flicker performance for the driving patterns by using a deep learning model and then select the patterns associated with desired phase flicker levels. The proposed model is simple but effective, and it does not require high computation power. Depending on the training data fed into the model, it is expected to work for various LCOS devices with different LC mixtures. We also aim to provide a comprehensive technical guideline for phase flicker optimization, in terms of the clock frequency of the LCOS backplane circuit and the LC viscosity, to help the end users gain a better understanding and improve the phase resolution of the LCOS devices to optimum for their desired applications.
2.1. Optical Setup
The total phase modulation depth of the phase-only LCOS device operating in a reflective mode is given by  where is the wavelength, is the effective refractive index and is the ordinary refractive index, and is the layer thickness. The phase response of the LCOS devices cannot be directly measured but can be obtained by measuring the intensity response. The phase flicker is defined as the standard deviation of the temporal phase fluctuation.
The optical layout of the test rig is shown in Figure 1. The system is capable of operating in both diffraction-based mode and crossed polarizer-based mode, both of which can be used to characterize the phase flicker performance of the LCOS devices.
When operating in a diffraction mode, the polarization direction of the incident beam is aligned with the LC alignment direction of the LCOS device by polarizer P1 and half-wave plate without using polarizer P2. With the incident beam shining at a fixed point at the middle of the active area of the LCOS device, the output intensity is recorded by the photodetector when a binary grating is displayed. The relationship between the phase modulation depth and the intensity in this mode can be obtained from the equation of diffraction efficiency, which is defined as the ratio between the power of the light appearing in a single diffraction order and the intensity of the incident beam. The diffraction efficiency of the th diffraction order of a binary grating can be expressed as  where is the peak-to-peak phase difference of the grating. The first-order diffraction power can then be derived using Equation (2) as where is the gray level. As can be noticed in Equation (3), the maximum first-order efficiency of (~40.5%) occurs when the phase modulation depth is . Hence, Equation (3) can be rewritten as where is the maximum first-order diffraction intensity. The phase modulation depth is then
When operating in a crossed polarizer mode, the incident beam is linearly polarized by the first polarizer P1, which is crossed with the second polarizer P2. The LC alignment direction is 45° with respect to the crossed polarizers. The output intensity is recorded when a bitmap (with one gray level uniformly applied to all the areas in the bitmap) is displayed on the LCOS device in this mode. The phase modulation depth in terms of the intensity has been derived in the previous work as  where is the normalized intensity.
2.2. Digital Driving Method
The digital PWM method is implemented to drive the LCOS devices used in this work, which were assembled in the group  based on JDC SP55 digital backplanes . By toggling between two boundary voltages, any intermediate RMS voltage can be represented to drive the liquid crystal. Different levels of the applied voltage are generated by changing the total duration of ON (i.e., logic 1) and OFF (i.e., logic 0) segments among a set of pulse segments distributed over a time interval.
The driving waveform at each gray level can be extracted as a PWM waveform and is determined by two configuration files, i.e., WPMap and LUT. WPMap initializes the structure of the driving pattern by creating several time segments over one frame time, while LUT specifies the detailed sequence by determining whether the individual time segment is switched ON or OFF during operation.
The driving pattern is defined as the part of the driving waveform within one frame time. Figure 2 illustrates an example of how to extract such a pattern at a certain gray level from the driving configuration files. One frame time contains 4 bits, and the length of each bit (i.e., ) is customized to have 128 time slots, which gives the driving pattern a total length of 512 time slots. A time slice smaller than one-bit time (i.e., ) is also introduced to increase the driving frequency. WPMap defines the lengths of and as well as the state of the time difference between these two (i.e., ), which is either ON or OFF. LUT defines the state of , and only the LUT for gray level 1 (i.e., ) is shown in this example.
The frame time is the sum of the total time slots allocated to all bits and is given by  where is the total time slots within one frame, is the number of the screen write pointers, is the number of memory clocks allocated to a row write, and is the memory clock frequency.
The WPMap used in this work is designed to have 26 bits within one frame time, and the length of each bit is 1080 time slots which matches with the LCOS resolution of 1080 pixels in its vertical dimension; i.e., the total time slots equal to . The driver has a frame time of 2.2 ms in this work, with a of 28,080, a of 1, an of 16, and an of 204 MHz.
3.1. Phase Flicker Optimization by Deep Learning
3.1.1. Prediction of the Phase Flicker Performance
The aim is to firstly build a predictive regression model to predict the phase flicker values for the given driving patterns with certain temporal pulse arrangements. Subsequently, select the pattern which has the smallest predicted value among all the possible patterns at each gray level, to form an LUT that can achieve the best phase flicker performance.
(1) Preparation of the Dataset. The WPMap used here produces a selection of up to 1344 gray levels, with a total number of more than 67 million possible driving patterns of different temporal pulse arrangements. 20 LUTs are randomly generated, each of which contains different switching information for 256 (8 bits) gray levels over a phase modulation range of . The structures of the above WPMap and LUTs are shown in Figure 3. Phase flicker values for these 5120 () driving patterns are derived by the phase values obtained in Equation (6) from the corresponding output intensities, which are experimentally measured by the crossed polarizer setup. Since the number of the available patterns at some gray levels (i.e., the gray levels close to 0 and 255) is less than 20 because of the limited time segment arrangement in WPMap, phase flicker values with the same driving patterns are averaged to eliminate the effect of duplicate cases. The cleaned dataset has 4874 pairs of data, where phase flicker values are the predicted outputs and their corresponding driving patterns are the inputs.
(2) Input Features of the Model. Each temporal driving pattern is constructed by 28,080 time slots, which gives a large dimension of input attributes. An example of the approach to reduce the input dimension is illustrated in Figure 4, by making use of the relationship of WPMap and LUT. After performing an element-wise multiplication between and , the dimension of the input driving pattern at gray level 0 has been reduced from 28,080 to 26.
(3) Fully Connected Deep Network. A fully connected deep network (an example diagram of such network architecture is illustrated in Figure 5), which consists of five fully connected layers, is constructed to build the predictive regression model for the prediction of the phase flicker. The five fully connected layers contain 80, 120, 20, 10, and 1 neurons. Except for the output layer, the activation function of each layer is .
3.1.2. Model Evaluation
The hyperparameters of the neural network model described in the previous section are set during the training phase with a validation set. In order to evaluate the performance of the model, 5-fold crossvalidation is conducted. The dataset is randomly partitioned into 5 subsets. In the first iteration, the first subset is used to test the model and the rest is used to train the model. In the second iteration, the second subset is used as the testing set while the rest serves as the training set. This process is repeated until each subset has been used as the testing set. As the subsets are randomly selected, the mean squared error (MSE) can vary slightly between runs. Hence, their average values are reported as follows: the average of the MSE is and the average of the standard deviation of the MSE is .
Figure 6 shows the visualization of the model performance, where the vertical axis stands for the phase flicker values predicted by the model and the horizontal axis represents the corresponding experimental phase flicker values. The red line indicates the perfect prediction scenario; i.e., all the predicted values match with the experimentally measured ones with zero error. To determine how good the predicted values match with the red line, the coefficient of determination, i.e., , is calculated. The prediction has an of 0.9492 to the 100% accurate case.
3.2. Practical Implementation of 10-Bit Phase Modulation
For the default 8-bit configuration, each of the color channels (i.e., RGB) of the driver has a resolution of 8 bits. 10-bit modulation of the current driver can therefore be physically enabled by using two of the channels together. The 10 bits are separated into 8 bits provided by the red channel and 2 bits provided by the green channel as shown in Figure 7.
An example of generating an bitmap with a gray level of 690 is demonstrated. The decimal number of 690 is firstly converted into its binary representation of 1010110010. The information in the first two of the most significant bits is isolated for the green channel, and that in the other bits is for the red channel. The blue channel is unused and left with zeros. A combination of red and green can be clearly noticed in the 10-bit bitmap created by such a strategy.
The LCOS device used here has a resolution of pixels with a pixel pitch of 6.4 μm. Phase fluctuation measurements at and are shown in Figure 8(a) for 8-bit modulation with default driving settings and Figure 8(b) for 10-bit modulation with the driving waveforms optimized by the deep learning model. 10-bit modulation has been physically realized with 1024 gray levels over the phase range of . Compared to the default driving waveforms which produce a mean phase flicker of over all the addressed gray levels, the driving waveforms optimized for 10-bit modulation in this work have reduced the phase flicker by three times to . The phase flicker level is low enough to be at a similar level as the one produced by analogue LCOS devices, i.e., typically , although analogue LCOS devices are traditionally associated with better performances in terms of the phase flicker, compared to the digital ones.
To further investigate how good the adjacent gray levels can be separated without overlapping, separation probability  is calculated by Equation (8) is plotted in Figure 8(c). where is the average phase flicker over all addressed phase levels, is the mean value of the phase fluctuation at a certain phase level, and is the one phase step for -bit modulation over a modulation range. For the 10-bit modulation, there is a chance of 62.63% for the phase to stay within the correct level, which is quite comparable with the probability of 73.56% for the default 8-bit modulation. The spatial information bandwidth of the LCOS device is hence improved by 25%, from ~1.6 Gb/sec to ~2 Gb/sec.
As 10-bit modulation is enabled by borrowing 2 bits from the green channel, it is in principle possible to achieve even higher resolution for the phase modulation by utilizing all the available color channels, as long as the level of the phase flicker can be further minimized to guarantee enough separation between adjacent phase levels.
3.3. Technical Guideline for Phase Flicker Optimization
Based on our understanding, apart from the LC driving waveforms, there are at least two other factors which can largely affect the level of the phase flicker. The first one comes from the hardware side, which is the clock frequency of the LCOS backplane circuit. The clock frequency determines the minimum achievable time segment in the signal, which subsequently affects the length of the pulses in the driving waveform and hence the phase flicker. The second one comes from the material side, which is the viscosity of the LC material used. Increasing the LC viscosity can result in an increased damping force and hence a smaller phase fluctuation over time. In practical measurements, the LC viscosity can be controlled by altering the device temperature, and the clock frequency can be translated into unit time length of the driving signal. A technical guideline will demonstrate the phase resolution performance (i.e., phase modulation level) in terms of circuit clock frequency and LC viscosity.
3.3.1. Temperature-Dependent Phase and Intensity Fluctuation
Temperature-dependent intensity response and the corresponding phase response for two LCOS devices filled with the same LC mixture are obtained at by binary diffraction. The fluctuation of phase and intensity is then averaged over the modulation range of for both its peak-to-peak difference and its standard deviation to get the flicker information. The measured data is normalized at 60°C and is shown in Figure 9. The relationship between the fluctuation and temperature can be linearly fitted as the dashed line—the phase flicker (i.e., standard deviation of the phase fluctuation) linearly increases as the temperature increases.
3.3.2. Temperature-Dependent Viscoelastic Coefficient
Phase decay measurement described in  is used to determine the viscoelastic coefficient (, where is the rotational viscosity and is the splay constant) of the LC material. When a small bias voltage which is not far from the threshold voltage of the LC is removed instantaneously, the transient phase change of a parallel-aligned LC cell can be approximated as where is the net phase change for the switching from to and is the free relaxation time of the LC layer with thickness and is given by
Such transient phase change is measured with the crosspolarizer setup, at for different temperatures with a bias voltage of 1.305 V. It is worth mentioning that although the wavelength is different from the one used in other tests (i.e., 1550 nm), it should not affect the result as the viscoelastic coefficient is an intrinsic material property. The measured data is shown in Figure 10, and exponential decays can be observed.
The viscoelastic coefficient can then be calculated as where is the slope of as a function of time. Knowing that the LC thickness , the temperature-dependent viscoelastic coefficients can be found and are plotted in Figure 11. The result at is in good agreement with that stated in the LC mixture datasheet (the name and manufacturer of the mixture cannot be disclosed due to commercial restrictions). We can see that decreases as the temperature increases.
3.3.3. Unit Time-Dependent Phase Flicker
Unit time length is defined as the minimum time segment in a driving signal. The shorter the unit time length is, the higher the circuit clock frequency is. Unit time-dependent phase flicker behavior is measured by single pulse driving patterns at with the crosspolarizer setup. Driving patterns with only a single pulse but different pulse widths (i.e., duty cycle) are used in the measurements. The measured data is normalized at 2133 ns and is shown in Figure 12. A linear relationship can be observed between the phase flicker and the unit time length. Given the fact that if there is no applied voltage (i.e., the pulse width is equal to zero), there is no phase flicker, the fitting curve should be assumed to pass the origin. The relationship can be described as where is the phase flicker, is the proportional constant, and is the unit time length. The small offset in Figure 12 is a result of background noise in the measurement at low voltages.
3.3.4. Technical Guideline
The phase modulation level in terms of circuit clock frequency and LC viscosity can be derived based on all the data obtained in the previous sections by the following steps: (1)Find at which temperature the desired LC viscosity can be reached based on the relationship obtained in Figure 11(2)Find the corresponding normalized phase flicker at (°C) based on the relationship obtained in Figure 9(3)Use the experimental phase flicker data at 30°C with a unit time length of 78.4 ns obtained in Section 3.2 to unwrap the normalized phase flicker as , and then, find the proportional constant in Equation (12)(4)Given a desired modulation level bit and a desired separation probability , find the required phase flicker level by Equation (8)(5)Find the required unit time length with and by Equation (12)(6)Find the clock frequency with by Equation (7)
Assuming a separation probability of 70% and an of 16 (i.e., the minimum achievable value of the driver), the performance of phase modulation level is plotted in Figure 13. We can see from the plot that for a given circuit clock frequency, the higher the LC viscosity is, the higher the phase resolution can be achieved, although with a trade-off of the device switching speed. Practically, for the same LCOS device, the viscosity property can be controlled by the factors like device temperature and LC layer thickness. If the viscosity remains the same, then increasing the clock frequency will decrease the level of the phase flicker and hence linearly improve the bit resolution of the modulation.
From Figure 13, the modulation level in terms of the circuit clock frequency and LC viscosity can be expressed as where is the modulation level in a linear scale, is the scaling factor as a function of the LC viscoelastic coefficient as shown in Figure 14, and is the circuit clock frequency.
A fully connected deep network consisting of five fully connected layers was built to predict the phase flicker performance given a certain driving pattern. The model was evaluated by 5-fold crossvalidation, having an averaged MSE of and an averaged standard deviation of the MSE of .
10-bit phase modulation was physically enabled by using two color channels of the existing driver. Phase fluctuation was measured at and by using the driving waveforms selected by the deep learning model. A phase flicker of 0.055% over 1024 addressed phase levels in respect to the total modulation range of 2π with a separation probability of 62.63% was achieved, which proved the practical realization of the 10-bit modulation. The spatial information bandwidth of the LCOS device was hence improved by 25% from ~1.6 Gb/sec to ~2 Gb/sec at a frame rate of 100 Hz.
A comprehensive technical guideline for phase flicker optimization was provided, in terms of the clock frequency of the LCOS backplane circuit and the LC viscosity. For a given clock frequency, the higher the LC viscosity is, the higher the phase resolution can be achieved, although with a trade-off of the device switching speed. Practically, for the same LCOS device, the viscosity property can be controlled by the factors like device temperature and LC layer thickness. If the viscosity remains the same, then increasing the clock frequency will decrease the level of the phase flicker and hence linearly improve the bit resolution of the modulation.
The focus of future work can be on how to further suppress the phase flicker to achieve even higher phase resolution and hence the spatial information bandwidth. Other algorithms can also be considered for the phase flicker prediction model, in order to have better accuracy.
Conflicts of Interest
The authors declare no conflicts of interest.
Y. T., M. P., and D. C. conceived the concept for the experiment. Y. T. and M. P. performed the experiments and simulations. Y. T. wrote the main manuscript text. All the authors reviewed and revised the manuscript. D. C. supervised the work.
Y. T. would like to thank the Jardine Foundation for a Ph.D. scholarship. This work was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) through the EPSRC Centre for Doctoral Training in Integrated Photonic and Electronic Systems (EP/L015455/1).
- J. Li, Q. Smithwick, and D. Chu, “Scalable coarse integral holographic video display with integrated spatial image tiling,” Optics Express, vol. 28, no. 7, pp. 9899–9912, 2020.
- J. Chu, D. Chu, and Q. Smithwick, “Encoding and multiplexing of 2D images with orbital angular momentum beams and the use for multiview color displays,” Research, vol. 2019, pp. 1–11, 2019.
- A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Transactions on Graphics, vol. 36, pp. 1–16, 2017.
- H. Yang, B. Robertson, P. Wilkinson, and D. Chu, “Small phase pattern 2D beam steering and a single LCOS design of 40 1 × 12 stacked wavelength selective switches,” Optics Express, vol. 24, no. 11, pp. 12240–12253, 2016.
- T. Lu, N. Collings, B. Robertson, and D. Chu, “Design of a low-cost and compact 1 × 5 wavelength-selective switch for access networks,” Applied Optics, vol. 54, no. 30, pp. 8844–8855, 2015.
- W. J. Hossack, E. Theofanidou, J. Crain, K. Heggarty, and M. Birch, “High-speed holographic optical tweezers using a ferroelectric liquid crystal microdisplay,” Optics Express, vol. 11, no. 17, pp. 2053–2059, 2003.
- J. E. Curtis, B. A. Koss, and D. G. Grier, “Dynamic holographic optical tweezers,” Optics Communication, vol. 207, no. 1-6, pp. 169–175, 2002.
- Y. Tong, M. Pivnenko, and D. Chu, “Improvements of phase linearity and phase flicker of phase-only LCoS devices for holographic applications,” Applied Optics, vol. 58, no. 34, pp. G248–G255, 2019.
- H. Yang and D. Chu, “Phase flicker in liquid crystal on silicon devices,” Journal of Physics: Photonics, vol. 2, no. 3, article 32001, 2020.
- Z. Zhang, Z. You, and D. Chu, “Fundamentals of phase-only liquid crystal on silicon (LCOS) devices,” Light: Science & Applications, vol. 3, no. 10, article e213, 2014.
- J. Goodman, Introduction to Fourier Optics, Roberts & Company Publishers, 3 edition, 2005.
- Z. Zhang, A. M. Jeziorska-Chapman, N. Collings et al., “High quality assembly of liquid crystal on silicon (LCOS) devices for phase-only holography,” in Emerging Liquid Crystal Technologies V, vol. 7618, San Francisco, CA, USA, 2010.
- Jasper Displays Corporation, “JD2552 SP55 – Jasper Display Corp.,” http://www.jasperdisplay.com/products/wafer/jd2552-sp55/.
- J.-P. Yang, F.-Y. Wu, P.-S. Wang, and H.-M. P. Chen, “Characterization of the spatially anamorphic phenomenon and temporal fluctuations in high-speed, ultra-high pixels-per-inch liquid crystal on silicon phase modulator,” Optics Express, vol. 27, no. 22, pp. 32168–32183, 2019.
- S. Corporation, “Liquid crystal based spatial light modulator full high definition model/SLM-200,” https://www.santec.com/dcms_media/other/components_SLM-200-C-E.pdf.
- S. T. Wu and C. S. Wu, “Rotational viscosity of nematic liquid crystals a critical examination of existing models,” Liquid Crystals, vol. 8, no. 2, pp. 171–182, 1990.
Copyright © 2020 Yuan Tong et al. Exclusive Licensee Beijing Institute of Aerospace Control Devices. Distributed under a Creative Commons Attribution License (CC BY 4.0).