Heavy Data Clouds Over the Meteorological World Due to Satellite IoT

by Hub Urlings, Herbie Rijndorp and Marco Franken

Amsterdam, the Netherlands, May 4, 2020--Recent developments in IoT connectivity and the coming of ubiquitous global low cost IoT connectivity are bound to cause a major technology disruption in the meteorological  world.  

New types of terrestrial networks and in particular the new small satellite (< 500 kg) based IoT systems. Especially communication satellites in the nano sat category (1-50kg), called cube sats for their 10x10x10 cm based frames, will connect millions of weather sensors all over the world. These Low Power Global Area Networks (LPGAN), will disrupt the weather data gathering segment streaming big clouds of meteo sensor data back to ever more advanced meteorological models running in the cloud. The huge increase in the number of weather data points will not only increase the reliability of traditional macro forecasts for the country, but increasingly also enable improved meso forecasts (When and where will the hurricane hit the shore?) and micro forecasts (How does the weather this season affect the crops on my land?).  

Where the advances in weather technology occur throughout the whole process of data gathering, data analytics and modeling, forecasting and the user interface displaying weather reports, we will limit this article to the developments in data gathering.  In this article we will have a look at the impact of the new generation of satellite IoT system on the weather data gathering and how it will disrupt the meteorological world.  

“Making the Weather”

To make a forecast, meteorologists collect data from a wide range sources and feed those into weather model, essentially a big software program that runs on a supercomputer.  Multiple times a day, new atmospheric data — obtained from a mix of weather satellites and on the ground weather stations — is automatically fed into a model, which generates a prediction by applying equations from fluid dynamics, physics and chemistry to data like temperature, wind direction and humidity.  

Historically is was the government that made the investment for weather satellites and to build out a ground network of weather stations to collected weather data. Weather forecasts were a public service to society. Forecasting the weather and climate changes and distributed via national TV. The large investments for this e.g. for weather satellites are made from public funds and operated under control of public organizations like EUMETSAT and National Oceanic Atmospheric Administration (NOAA). 

Traditionally weather data comes from publicly owned weather stations on the ground (and seas) and doppler radar stations reinforced by weather balloons, sounding rockets, and more recently drones.

According to WMO, well over 10 000 manned and automatic surface weather stations, 1,000 upper-air stations, 7,000 Voluntary Observing Ships (VOS), 100 moored and 1 000 drifting buoys, hundreds of weather radars and 3,000 specially equipped commercial aircraft currently measure key parameters of the atmosphere, land and ocean surface every day.

In the UK alone in 2015 there were 330 UK weather stations, 16 weather radars and up to 300 ships and aircrafts, providing 106 million weather observations a day. To analyze these millions of observations, data flows to the European Centre for Medium-Range Weather Forecast (ECMWF) in Reading (UK), where one of the most powerful supercomputers in the world puts them into weather models and produces global numeric weather forecasts for users worldwide. 

Add to this ground based data gathering infrastructure some 16 meteorological and 50 research satellites to get an idea of the size of the global network for meteorological, hydrological and other geophysical observations. Once collected, observations are quality-controlled, based on technical standards defined by the WMO Instruments and Methods of Observation Programme (IMOP), then made freely available to every country in the world through the WMO Information System (WIS).

The data coming from these sources are often publicly available for private weather companies that then employ in-house teams of meteorologists and data scientists to re-skin and repackage the data and sell it to clients whose survival and risk-management depends on precision forecasts.  

Weather for Cash


Where weather services are traditionally provided by governments and public organisations for free as a form of soft-diplomacy, new commercial entrants are entering the market to both augment and extend weather data gathering and forecasting’s abilities

Weather is one of the biggest business opportunities in the world and grows by the day, both on macro and meso level. This applies for the macro level forecasts for a country. With the growing number of smart phones, lot of weather-related start-ups started providing intuitive, engaging and easy to use apps for weather data. The daily news forecast on TV is now joined by individual local weather reports, with billions of people now checking their (local) weather every day on their phone whenever they wish or need.

Growing more rapidly however is on the meso level of forecasts, triggered by weather induced emergencies like flooding, hurricanes and thunderstorms, flash floods, tornadoes or heath waves.

Extreme weather kills more than 500 people in the US alone. The impact of weather on the world is enormous, with 1/3 of our global economy being weather sensitive with a US$ 2.1 trillion impact of on the global GDP variation. This while considering that 5 billion people have limited or no access to weather forecasts. The cost of flooding and storms in the midwest of the US alone in spring 2019 alone was an estimated US$ 3 Billion. It is estimated that each year, the global trucking industry loses upto 32.6 billion trucking hours globally to weather-related events.

Weather causes 28 per cent of highway crashes, 19 per cent of fatalities and 23 per cent of delays across the globe.
This financial risk is why insurance companies, public government organisations, commercial enterprises and house owners are all looking for accurate forecasts on meso level. It is therefore no surprise that we see a growing number of weather start ups that promise by using Artificial Intelligence (AI) to revolutionize weather forecasting and deliver more actionable forecasts.

More Weather Data from the Ground

More data mean more accurate forecasting, and seismic advances in weather data gathering technology have enabled the recent emergence of so called micro forecasts.  New sensor and communication technology have brought the industry’s ability to collect and analyze data from every corner of the world, to a new level and the end is not in sight yet.

Apart from the “Official” WMO certified weather stations, we see that an abundance of meteorological sensors has become operational including cellular systems, weather balloons, drones and ocean-going robots. 

What to think of an exotic new system called Saildrone that provides drone powered global weather forecasts using ocean going sail drones that are fitted with sea, sky and horizon cameras and sensors that take atmospheric and ocean measurements? The ambition is to launch 1000 saildrones for planetary coverage at a 6x6 degrees resolution or one saildrone per 580 km square. 

 Weather Underground (part of The Weather Channel since 2012) alone has 250000+ private weather stations worldwide that share their data as a basis for weather forecasts. When you look at the distribution of stations in the graphic below, you see a one-on-one coverage map of terrestrial telecom networks. Under the header “Citizen science” Weather Underground is a global community of people connecting data from environmental sensors like weather stations and air quality monitors so we can provide the rich, hyper local data you need to power your passions. They use an open source model where contributors of weather data can also use the accumulated data sets for their own forecasts.
New data also come from the use of cellular based systems. Cellular IoT data can be used as one input for micro-cell weather forecasting, using the power levels of radio waves between the SIM and the cell tower to establish real-time weather conditions at an extremely local level.

                           Distribution of Weather Underground weather stations.                                                   (source: Weather Underground)

This information can then be combined with other data sets, such as telematics, to establish whether cars in the area have their lights on, or are using their windscreen wipers, building a more complete picture of weather conditions.

Commercial weather company Climacell claims to map the world’s weather data with millions of observations others can’t see. For their forecasts they see everything around them as a possible weather sensor, from wireless networks to connected cars. With so many sensors out there, they claim to have built a proprietary big data collection and analysis platform, combined with exclusive weather models. With this foundation, Climacell aims to provide minute-by-minute, street-by-street forecast, globally.

Using weather-related sensor data from smart phones, a Google patent seeks to predict ‘micro-climates’ (and maybe even map your commute away from inclement weather).  A patent from 2017 outlines a way for Google to use sensor data from mobile devices to determine and forecast up-to-the-minute weather conditions at “micro-locations.” The invention could make it possible for Google to predict “micro-climates” at sites as specific as a certain spot in the park, or a particular street corner of a city block.

More Weather Data from Space

Although an impressive number of weather satellites are in space, many satellites are 20 to 30 years old, carrying very relevant, but outdated technology.  Upgrading the weather satellite fleet is an expensive and time consuming enterprise but steady progress is made.  Billions of Euros are pouring into “space weather” with a global market projected to be worth between Euro 4B and 6B by 2025.
The Geostationary Operational Environmental Satellite is a good example (GOES 16 and Goes 17 satellites that have recently been launched) from the US by NOAA.  Instruments aboard the GOES satellites collect data through a Visible and Infrared Spin Scan Radiometer (VISSR), a high technology device that measures the vertical structure of water vapour and temperature in the atmosphere over specific locations. 
This allows the following data to be collected:

  • Images of cloud and snow formation
  • Wind information from cloud motion.
  • Estimates of precipitation.
  • Air and sea temperature.

In addition a new generation Polar Orbiting Environmental Satellites (POES) is coming into service. These satellites circle the earth at approximately 515 to 528 miles (830 to 850 km) in a North/South orbit sending regular images back to earth. This enables the onboard instruments (VISSR) to monitor and record global atmospheric conditions and cloud data by collecting numerous weather parameters that affect weather patterns.

From Europe the Copernicus constellation is a state-of-the-art weather satellite system developed in a cooperation between the EC and ESA with the objective to manage the environment and to understand and mitigate the effect of climate change and ensure civil security.
The fast-changing climate and economic developments around the world have created increasing demand for high-quality weather and climate data. At the same time, traditional weather satellites have become more expensive and have long development cycles. Is the small satellite revolution  offering an answer–constellations of small satellites, cooperating to produce the most precise, abundant and timely data on our planet?

Small sat start ups such as Spire Global, GeoOptics and later also Planetiq are in varying levels of development and operations, providing a images and weather data from inexpensive small sats in Low Earth Orbit, using satellite weather observations based on innovative sensor readings such as radio occultation. 

These space-to-(digital)-cloud analytics companies are now using their constellation of small satellites for entering the meteo market. Spire recently launched a Weather Forecast product, offering new weather forecasts dedicated to the maritime industry. The product will feature atmospheric and oceanographic weather attributes including, among others, sea surface temperature, ocean currents, wave height, surface wind, and air temperature, all available globally.  Time will tell if they will be able to successfully compliment existing government weather assets in a sustainable way.

New satellite IoT systems will have a dramatic impact on weather data gathering as, due to the low costs of IoT connectivity and the use of low power satellite terminals, the number of weather data points around the world will grow in an accelerated pace. This will enable massive data streams from meteo sensors all around the world to come together and augment weather observations derived from weather satellites and the ground weather station network. The use of satellite also allows us to fill the geographical gaps that currently existing in the existing ground infrastructure.

The Rise of Small Satellite IoT in Low Earth Orbit

A new generation satellite IoT systems that is using cube sats in Low Earth Orbit are emerging to provide low costs and low power global connectivity to a wide range of sensors including weather stations. Their connectivity cost will match the costs for terrestrial networks like Lora or Sigfox and will be in the range of Euros per month per data point.

Globally, half a dozen satellite IoT players from all over the world are working to get their small sat constellations operational and enter the market.  The current status in the LPGAN market is that the development of the end-to-end network is in its final phase. Among the is Amsterdam-based Hiber which has currently two satellites in low earth orbit and ground stations in Delft and Svalbard in the Netherlands.

All have launched and tested prototype satellites, ground stations have been established and ground modems and antenna equipment is available to be connected to a wide range of weather sensors. The satellite IoT players are now preparing for launch, that is expected end 2020 -  2021 to bring a number of new IoT connectivity services. 

The increase of data streams into advanced weather models running on cloud based supercomputing power means we can provide better and more accurate forecasts. 

This not only might improve our quality of life here on Earth but it is also opening up a completely new segment of the weather market. With the increasing availability of cost-effective data and imagery derived from small satellites, new opportunities are emerging to create applications that are tailor made to the needs and desires of public organisations, companies and consumers. 

The New Weather: Micro Forecasting

The number of applications derived from satellite IoT data streams looks endless and will vary between analysis of shipping traffic, optimizing energy grid management, creation of base maps for urban planners, natural hazard and disaster planning and recovery, airplane tracking, water productivity tracking, estimating surface water reserves, monitoring illegal logging and deforestation, and detecting underground water and mineral sources.

The meteorological industry today and tomorrow is fresh, young and hi-tech, especially in the new micro level forecast area:

  • Farmers need to know the optimal time to plant corn before a frost.
  • Airlines need to optimize their flight schedule in advance of a blizzard.
  • Walmart needs to plan pre-hurricane inventory
  • NGO’s to prepare for humanitarian emergencies at rapid response, micro level.
  • Cricket coaches now can match data about the pitch with atmospheric conditions:  ‘Big Data meets cricket.’
  • Shipping companies need weather forecasts for their immediate impact on their ships and as a new way to illuminate risk on maritime routes, conserve fuel, and safely increase productivity.
  • Energy companies need to optimize their solar panel location placement.

We will explore the above in a follow up article.

Conclusion

The data-gathering technology for weather forecasting is undergoing rapid growth. We see a range of new media being used for ever more accurate and localized forecasts, not only in the ground sensor networks, but also in space where an increasing number of weather satellites provides more and detailed weather observations. 

Small sat IoT satellites provide low cost connectivity to globally dispersed weather sensors now collecting data from locations that were previously unavailable due to high costs and availability. 

To bring them into use for the meteorological sector a couple of challenges need to be matches. The first is to choose between the different new IoT connectivity services offered by the various systems, that each have their own satellite terminal equipment and service specification.

The second is how to transform the incoming data into valuable information. 
To meet those challenges we have founded  https://SatIoTLab.eu were expertise and best practises on existing and new satellite IoT connectivity is gathered and distributed via webinars, workshops and publications aimed specifically at public and government sectors like the meteorological world. 

The new flood of weather observation data this will bring will not only make the macro forecasts more accurate, it also provides “any-time” weather forecasts that are tuned to specific personal and professional requirements.  We are not yet so far that you can get the weather you like, but you can rely on the forecast of the weather you get. 

-------------------------------------------

Hub Urlings was one of the pioneers of Satellite M2M as Product Manager Inmarsat-C at the famous KPN Station 12. The success of this “small data” satellite service, its global coverage and reliability made that the service was used for a myriad of applications: from sending messages, to truck fleet management, to pipeline monitoring and bringing back data from all types of sensors.  At that time satellite was the only type of network that was able to offer global coverage for what we would now call IOT services.  Now, 25 years later he is again involved in the development of a new generation of smallsat based Sat-IoT services as ESA Innovation Manager at Hiber.global.  To meet the complexity of the sat-IoT value chain he developed the SatIoTlab.eu as an education and co-creation platform for global sat-IoT applications. He can be reached at urlings@m2sat.com

Herbie Rijndorp joined the International Maritime Satellite Organisation (Inmarsat) in London during the pre-operational phase in 1981. He worked at application development and introduced Electronic Data Interchange (EDI) for Machine-to-Machine data communication over the end-to-end Inmarsat links. Later-on he worked on EU sponsored projects developing Remote Monitoring applications via the Inmarsat 64 kbps and 384 kbps High Speed Data links.  Herbie also established the Quality of Service department at Inmarsat, measuring in real time the performance of the communication links on the Inmarsat end-to-end links.  Later he was one of the founders of M2sat and as such currently working on the start-up of the M2sat-IoT lab.  He can be reached at rijndorp@m2sat.com

Marco Franken has worked at Philips developing early stage APIs for aero and government networks, followed by service and applications development at Inmarsat. In 2001 he managed introduction of GPRS data services in Germany and UK, as well as developing multi channel platforms for extreme environments. From 2006 he worked with Sybase and SAP to provide global availability of data messaging.  More recently, his focus has been the efficient use of sensors, IoT and intelligent data across insurance, finance, farming and maritime. He can be reached at franken@m2sat.com