Posts

Blickfeld exhibition booth CES 2020

This was CES 2020

Microelectromechanical system - MEMS technology

At the heart of sensors – MEMS technology for LiDAR

“MEMS-based LiDAR sensors are usually less expensive, but they are not high-performance enough for use in autonomous vehicles.” We hear sentiments like this quite often. In this blog post we will explain how our sensors effectively invalidate this assumption, how we found the perfect mirror size for our LiDAR, and what considerations factored in to our decisions.

For use in autonomous vehicles, LiDAR sensors have to meet two basic requirements: on the one hand, they have to deliver high performance, including long range and a wide field of view. On the other hand, they must also be scalable so that millions can be produced and installed in vehicles. LiDAR manufacturers meet these challenges through various approaches. Mechanical LiDAR systems, whose beam deflection units are moved mechanically by motors, are still the most commonly used systems today. Although these devices boast a wide field of view — some up to 360° — and long range, their mechanics require regular maintenance and are large, heavy, and expensive to produce. Thus, mechanical LiDAR systems solve only the performance side of the two major demands placed on the sensor industry.

Another approach to meet these challenges is MEMS (microelectromechanical systems) technology. Here, components are produced in silicon, which has the advantage of scalability: since this technology has been tried and tested over many years, identical components can be produced in a cost-efficient manner and in large quantities. This approach is also used, among other things, in the production of microsensors.

But how do MEMS-based LiDAR systems meet the challenges of performance?

Long range with the help of the right laser source

For autonomous vehicles to be able to travel at high speeds, they must be able to “see” and perceive the world around them — not only in their immediate vicinities, but also at greater distances. This is particularly important when driving on highways, as vehicles are moving faster and therefore objects, bends, and other vehicles must be reliably detected at greater distances in order to be able to react in good time. Sensors therefore require a long range in order to enable autonomous driving at highway speeds.

In order to achieve this range with a LiDAR sensor, either the emitter or the detector needs to be optimized specifically for this application.

One possible starting point for such adjustments is the laser source. Typically, lasers with two different wavelengths are used in LiDAR sensors. Some LiDAR manufacturers rely on fiber lasers with wavelengths of 1550 nm. This wavelength cannot be focused by the human eye and can thus be used in an eye-safe manner even at high energy levels. This results in a longer range — the more energy used, the further the device “sees”. However, this type of laser source also has a decisive disadvantage: 1550 nm lasers are large and complex to manufacture, which leads to higher prices and large LiDAR housing dimensions.

Many LiDAR applications therefore use laser diodes that emit laser pulses with wavelengths of 905 nm. These have the distinct advantage of being very small and having been used for a long time in a wide variety of applications. As a result, these diodes are inexpensive and available on the market in large quantities. However, eye safety regulations require that the beam strength of the diodes be lower than that of 1550 nm lasers. The optimization on the emitter side is therefore limited.

 

Searching for the right mirror size

So how can the detector be optimized? Here the aperture plays an important role in achieving long ranges. It describes the size of the detector. In the case of our MEMS-based design, the aperture corresponds to mirror size. In order to capture as much light as possible, a large aperture — in other word as large a mirror as possible — is required. However, mirror size is also limited by certain factors – and so it is necessary to calculate the optimal mirror size while taking these into account. These factors are: photon number to be received, collimation, deflection angle, and resonance frequency.

Photon number

On the one hand, the size of the mirror used in LiDAR unit depends on how many photons have to be emitted in order for a sufficient number of photons to come back, thereby detecting an object. This minimum number of photons can be calculated accurately based on the link budget. This measure includes how many photons are lost at distance and through low reflective surfaces, homogeneous scattering of light, and detector inefficiency. In this way, it is possible to calculate how many photons must be emitted, or how large the aperture must be so that a minimum number of photons can be detected again. In addition, Blickfeld sensors have a coaxial design, which means that only the light that comes back from the same direction in which it was emitted is recaptured. This is advantageous in that it prevents other random light signals from being picked up and disturbing or falsifying the images.

Collimation

In order to obtain high-resolution data that reliably identifies even small objects, the lasers must hit objects in a collimated form. This collimation is achieved by placing a lens in front of the laser. Now the mirror size comes into play again: a mirror must be exactly large enough to deflect all the light collimated by the lens. This depends on the focal length required for optimal collimation and thus high resolution.

Resonant frequency

MEMS mirrors oscillate at a certain resonant frequency. They are triggered by integrated actuators and therefore do not require motors or any other mechanical means. This is a clear advantage because motors and moving parts quickly wear out and require regular maintenance. These problems do not arise if the oscillation is triggered by integrated actuators.

The resonant frequency at which a mirror oscillates depends on the size and mounting of the mirror. For this purpose we have developed a proprietary technique for embedding the mirrors in order to be able to use particularly large mirror sizes. Due to these unusually large diameters, a large number of photons can be directed into the surroundings and back onto the detector, which allows Blickfeld LiDAR sensors to achieve accurate a long range. In addition, thanks to their larger size, these mirrors are more robust than conventional products, which are only a few millimeters in diameter. Mirrors used in Blickfeld LiDARs also have high resonant frequencies due to their lightweight construction, which ensures that the greatest possible number of photons are returned to the detector: if the mirror oscillates too quickly or too slowly, photons will pass the detector due to the coaxial structure.

MEMS technology specifically designed for LiDAR applications

In conclusion, mirror sizes are determined by a wide array of factors. In order to build the most high-performance LiDARs based on MEMS, mirrors must have specially developed compositions, sizes, and embeddings. And only if the MEMS technology is specifically developed with LiDAR applications in mind, can the requirements of a long range, a wide field of view, and high resolutions be achieved.

 

 

 

How to reduce congestion in cities

Smart traffic with the help of LiDAR technology

If you have ever driven along the Mittler Ring in Munich during rush hour, you will be aware of the enormous problem facing urban spaces throughout the world: traffic jams as far as the eye can see. On average, Germans were stuck in traffic for 120 hours – and, in Munich, for as long as 140 hours – in 2018. For individuals, these lost hours are annoying and impair their quality of life. For the state, though, these figures mean considerable economic effects. Traffic jams cost several billion euros per year because employees are stuck in traffic jams instead of being productive, and goods are on the road instead of on the shelf. In addition, there is a high level of environmental pollution due to increased fuel consumption and thus increased CO2 emissions.

Traffic is a problem, especially in cities. However, new technologies can and will make an important contribution to solving this problem.

 

Knowing more than the individual road user

How does congestion develop? Traffic jams are a distributed problem caused by the fact that all road users drive their vehicles in a way that is optimized for themselves. For example, they may catch up with the car in front or change lanes – whatever appears to them to be the best way to get to their destination faster. Since the individual road users cannot see how they influence the traffic around them, they cannot act accordingly. Drivers themselves are not aware that a traffic jam can be triggered three kilometers behind the vehicle that is stopping.

This is the point at which we have to start: the behavior of individuals must be counterbalanced in traffic planning and control so as to optimize the flow of traffic. The solution is to regulate traffic in a pre-emptive and distributed manner, i.e. with anticipation and going beyond the individual road user. This requires a complete overview of the traffic situation.

Gaining a complete overview with GPS, cameras and sensors

Various technologies can be used to achieve this complete overview. However, if you take a closer look at them, you will see that some of them are less suitable for equipping a smart infrastructure:

GPS:

GPS provides valuable data by tracking the movements of road users. This technology can therefore be used to report traffic jams quite reliably. The ability to take pedestrians and cyclists into account, however, exceeds the capabilities of GPS.

Instead of collecting information with the help of road users, as in the case with GPS, sensors and cameras can be integrated into the infrastructure to monitor the traffic situation. This requires that the devices be installed in traffic lights, street lamps or traffic signs so that they can collect information about their surroundings from there. Here, too, clear advantages and disadvantages of the possible technologies can be identified:

Cameras:

Cameras, for example, are able to record color images, but they cannot provide the same quality when used in darkness. Additionally, they only capture the data in 2D, whereas 3D data is needed to reliably detect objects and determine distances. They also quickly find themselves in a grey area with regard to data protection when recording and storing personal data.

Radar:

Radar is mainly used for speed monitoring, but could also be used for traffic monitoring. However, radar only provides a very crude picture: although the technology identifies objects, it is not able to classify them due to the lack of detail. Radar data, for example, cannot reliably distinguish between pedestrians and cyclists.

LiDAR captures road users precisely and anonymously

Laser-based LiDAR technology is a technology that can distinguish very precisely between all road users. The sensors provide detailed and reliable 3D information that makes it easy to distinguish between different road users. Although it is possible to recognize whether the 3D point cloud is a pedestrian or a cyclist, the identification of individuals is not possible, which protects the privacy of road users.

In addition, LiDAR sensors are able to reliably collect information even in difficult weather and lighting conditions. Darkness, dust or fog do not bother the technology. In addition to position and object information, the sensors also record speeds, which can be helpful in analyzing traffic flow or the causes of traffic jams.

Solid-state as a solution for today’s LiDAR problems

High-tech sensors are currently used primarily in the field of autonomous driving, but they face a major challenge: the LiDAR sensors that are available today are expensive and prone to faults. Solid-state technology solves these problems. In this type of LiDAR, the moving parts that deflect the laser to scan the environment are replaced by maintenance-free components. The sensors are therefore much more robust and also less expensive – and hence ideally suited to a wide range of applications in the infrastructure.

Traffic information enables practical measures

The LiDAR sensors installed in the infrastructure provide information about the current traffic situation in real time: Is the traffic flowing or stagnating? Is there an accident or a construction site? Are there many pedestrians at the traffic lights or at the crosswalk?

With this information, the following measures can be taken in real time and adapted to the traffic in order to optimize the traffic flow:

  • Adaptation of traffic light phases
  • Adaptation of speed limits
  • Displaying traffic jam warnings
  • Showing redirection recommendations
  • Identification and reconstruction of hazardous locations

In future we will even go one step further: autonomous vehicles will then use the information to dynamically adapt their schedules and routes to the traffic situation.

Cities that revolve around people again

In many cities, the influences of the paradigm of the “car-friendly city” can still be clearly seen today: urban planning is aligned to the goal of unhindered traffic flow by car. Even though this model has been subject to strong criticism for several decades now, many traffic concepts in cities are still oriented towards motorized individual transport.

In recent years, this approach has been increasingly replaced by a demand for car-free zones or even entire city centers. These demands clearly show that urban and traffic planning must once again be more about people. The needs of residents, commuters and all other road users must be centermost, which means making mobility as safe and uncomplicated as possible. Pedestrian crossings must be designed to be safer; turning accidents must be avoided; sufficient space must be created for cyclists – the list of measures is long. Intelligent traffic control with the help of a smart infrastructure makes this possible – and LiDAR technology is at the heart of it.

LiDAR for everyone – MEMS mirrors enable LiDAR sensors for the mass market

In the past, LiDAR systems struggled with a number of problems:  they were lacking in efficiency and robustness for use in the automotive industry. An even bigger problem was the cost of a LiDAR system: The sensors were far too expensive for the automotive mass market. Blickfeld tackles this problem. The Munich-based start-up’s technology relies on MEMS silicon components and highly automated production – and is paving the way for autonomous vehicles.

The problem: lack of robustness and high production costs

LiDAR technology is not a new invention; it has been used in various applications, such as measuring emissions in atmospheric research or remote sensing in archaeology since the 1960s. However, LiDAR systems have faced some serious problems until now:

  • LiDAR systems are very large.
  • The mechanics used are sensitive to harsh environmental conditions such as vibrations, heat and cold.
  • The range of previous LiDAR systems is not sufficient for many applications.
  • And: LiDAR sensors today are neither available in large quantities nor at affordable prices.

All these points pose major problems for the automotive industry in particular, which is desperately looking for high-performance LiDAR systems with a robust and production-scaleable design. Why? Because LiDAR is essential for the autonomous cars of tomorrow. Mobility is about distances and speeds. Not-colliding is essential in road traffic – LiDAR sensors detect immediately and reliably. Experts therefore predict that several LiDAR sensors per car will be needed in the future. However, to achieve this, the problems mentioned must be overcome.

What is the root of those problems?

Classical, mechanically rotating LiDAR concepts are fragile and complex in design and production, resulting in large dimensions and high prices. Although other technologies and designs such as optical phased array and flash LiDAR have the potential to significantly reduce LiDARs in size and cost, they are still at the beginning of development or have system-related disadvantages, such as a very limited range.

The solution: MEMS-based LiDAR sensors

LiDAR sensors based on MEMS mirrors are a promising solution. Their silicon construction is already well advanced and implemented very successfully in many automotive applications. In addition, MEMS technology does not require rotating components and is therefore much more robust and durable than mechanical LiDAR systems. MEMS mirrors available on the market today have small mirror sizes and small deflection angles, as these were sufficient for their previous use. Their performance in terms of range and field of view is therefore limited.

MEMS mirrors on wafer level

The Blickfeld solution: Particularly large mirrors

Blickfeld’s LiDAR system relies on MEMS. But how does their system differ from other MEMS based LiDARs? To extend the range of the sensors, Blickfeld has developed its own MEMS mirrors. With generous dimensions of more than 10 millimeters mirror diameter, a high proportion of incident light can be directed onto the photodetector. Thus, the LiDAR reliably detects even weakly reflecting objects at a distance of more than 180 meters. In comparison, conventional MEMS mirrors have diameters of only a few millimetres and small deflection angles, which reduces the range and field of view accordingly when used in LiDARs.

Further advantages of the MEMS mirror used in Blickfeld products are:

  • a large deflection, which enables a scanning angle of approx. 100° x 30° and thus a wide field of view.
  • Spatial filter effect due to coaxial design: The light emitted by the laser and deflected onto the scene by the MEMS mirror is reflected by the objects in the field and returned to the detector along an optically almost identical path. This means that light photons are only collected from the exact direction in which the laser sent them. This minimizes the background light, for example from the sun or other LiDARs, and produces a very high signal-to-noise ratio, which benefits the range.

Blickfeld Cube LiDAR sensor mounted on a car roof

How does Blickfeld address the problem of production?

The MEMS mirrors developed by Blickfeld solve three of the four problems mentioned above: The company’s LiDARs are space-saving, robust and performant. But how does Blickfeld tackle the problem of cost-intensive production? By manufacturing the mirrors with low-cost photolithographic production techniques that allow highest precision with extreme scalability, so-called MEMS silicon manufacturing. In a highly automated process, a standard silicon wafer with a diameter of 200 millimeters is turned into hundreds of MEMS components simultaneously. This method, which has been tried and tested in the semiconductor industry for decades, enables the technology to conquer the mass market.

The MEMS mirror is embedded in a ‘commercial off-the-shelf’ structure, i.e. commercially available standard components. These laser and detector units, that are available on the market, enable a cost-effective and scalable production of the sensors.

Autonomous cars and so much more

The use of LiDAR sensors extends far beyond the automotive industry and the fields of application are versatile. In order to take advantage of the technology, Blickfeld has eliminated the major problems of LiDAR systems and has made them accessible to the mass market. The LiDAR era has only just begun!