Skip to main content

Accuracy Assessment Case Studies: Space Intelligence vs Open-Source Datasets

How the Space Intelligence EUDR maps compare with open-source datasets, including a test on real coffee farms in Brazil.

Written by Ferry Febrian

Based on the Space Intelligence ICE CoT EUDR Deforestation Maps overview (see Source at the end). For the accuracy assessment approach and the Cambodia case study, see Space Intelligence - How we make maps. For the EUDR forest and deforestation rules, see Understanding Deforestation under EUDR.

On 541 coffee fields in Brazil, the Space Intelligence maps passed 0% of non-compliant farms and wrongly failed 3% of compliant farms. Open-source datasets passed up to 65% of non-compliant farms (MapBiomas) or wrongly failed up to 14% of compliant farms (JRC GFC v2).

Why open-source datasets are not enough for EUDR

Open-source datasets are useful for monitoring forests at a global scale, but they are not the right choice for verifying EUDR compliance.

Maps such as Global Forest Change (Hansen et al.), Global Forest Watch and the EU JRC Global Forest Maps:

  • use optical satellite data only;

  • map tree cover, rather than following the EUDR definition of forest;

  • do not reliably separate natural forest from plantations or tree crops such as cocoa and coffee.

In the Ivory Coast 2020 example below, the open-source datasets show far more forest than Space Intelligence. Much of that "forest" is in fact cocoa plantations. Mapping land cover under EUDR needs precise local calibration and an understanding of how tree crops grow.

Source: Space Intelligence, ICE CoT EUDR Deforestation Maps overview, page 6.

Brazil case study: the datasets compared

Dataset

Pixel resolution

Built for EUDR?

Notes

JRC GFC v2

10 m (many inputs at 30 m or coarser)

Partly

Funded by the European Union to support EUDR compliance; attempts to follow the EUDR forest definition and the end-2020 cut-off date.

MapBiomas Collection 10

30 m

No

High-quality annual land cover dataset by Brazilian scientists (INPE and others); not designed to meet EUDR definitions.

Space Intelligence

10 m

Yes

Built from raw optical and radar satellite data specifically for EUDR testing, following the EUDR forest definition.

Source: Space Intelligence, ICE CoT EUDR Deforestation Maps overview, page 9.

How the test was done

  1. Random points (2025). An early assessment used 570 randomly distributed points across Brazil. Overall accuracy of the forest/non-forest maps was 95% for Space Intelligence, 92% for JRC GFC v2 and 91% for MapBiomas Collection 10.

  2. Real farms. Most of those points were not near coffee or cocoa farms, so a second test focused on what matters for EUDR: whether real farms are correctly passed or failed.

  3. Sun-grown coffee in Brazil. Large coffee fields can be identified with high confidence from high-resolution satellite imagery. Analysts traced each field boundary as a polygon.

  4. Expert check. Each field was checked for forest on or after 31 December 2020. This produced 541 coffee fields confidently classed as compliant or non-compliant.

Non-compliant coffee is rare in Brazil: of around 2 million hectares of coffee, only around 1,300 hectares is estimated to be on land deforested after 31 December 2020.

Source: Space Intelligence, ICE CoT EUDR Deforestation Maps overview, page 9.

Results

Dataset

Non-compliant farms passed (false negative)

Compliant farms failed (false positive)

Space Intelligence

0%

3%

JRC GFC v2

1%

14%

MapBiomas Collection 10

65%

12%

  • False negative = a non-compliant farm wrongly passed. This is the bigger risk for buyers, and Space Intelligence passed none.

  • False positive = a compliant farm wrongly failed. Space Intelligence is working to bring this below 1%.

Source: Space Intelligence, ICE CoT EUDR Deforestation Maps overview, page 10.

What if a compliant plot fails?

False positives will never be zero. For plots that fail in ICE CoT but that the submitter believes should pass, there is an Appeals Process: the Space Intelligence expert team reviews the plot using high-resolution satellite data and decides whether it should pass.

Protected forest inside Brazilian farms

None of the sampled fields contained EUDR-defined forest. However, Brazilian coffee farms are often legally required to keep areas of forest, and these are usually included inside the farm boundary. ICE CoT includes a tool to exclude these areas from the assessment. With open-source datasets, compliant Brazilian coffee farms would likely be flagged as deforested, making the differences above even larger.

Source: Space Intelligence, ICE CoT EUDR Deforestation Maps overview, page 10.


Source

Space Intelligence Ltd. Space Intelligence ICE CoT EUDR Deforestation Maps (Overview, 10 pages). space-intelligence.com

  • Page 6: A note on open-source forest and forest change datasets

  • Page 9: Brazil map assessment, datasets compared and how the test was done

  • Page 10: Results, the Appeals Process and protected forest inside Brazilian farms

The ICE CoT EUDR Deforestation Maps are available under licence from ICE Benchmark Administration Limited.

Did this answer your question?