Using AI to Understand Buildings

We’re very pleased to share that TheSqua.re has published another peer-reviewed scientific research paper. This work showcases our original approach to using AI.

What’s it all About?

As we explained in the previous research briefing, we have spent a long time creating EcoGrade, the industry-leading sustainability score in terms of its accuracy. To ensure our numbers are as good as we can make them, we use the best government-approved datasets we can find.

But sometimes, for some locations, there is no real data. Nobody knows how a building was constructed. Nobody has recorded whether it has insulated walls or double glazing. There is zero information about the type of heating or lighting it might have.

A complete lack of data is a problem for us because we pride ourselves on being as accurate as we can be. But we realised that we have something unique. Here at TheSqua.re, we have listing data comprising thousands of detailed photographs of the interior and exterior of rental accommodation. In some cases, we have excellent sustainability information corresponding to those apartments, and in other cases, we may not. This dataset is exactly the kind of thing today’s Artificial Intelligence likes to learn from.

So we got ourselves a state-of-the-art AI, and we put it to work. From photographs alone, could it figure out useful data? Could it tell us the possible age of the building? Could it look at radiators and work out whether the apartment is heated by gas or electricity? Could it distinguish between electric storage heaters and electric panels? Could it predict the likely percentage of low-power lighting? The answer was yes.

It took some work to teach the AI what to look for, and there’s always room for improvement. But we showed that even when we have no useful data – and we cannot look at neighbouring properties to help – an AI can study photographs and be remarkably accurate in predicting useful data, which we can then use to calculate EcoGrade scores and CO2 values.

EcoGrade Logo

How do we know it was accurate?

This is where the science comes in. We created a dataset of apartment photos where we knew the answers already but we didn’t tell the AI. Then we asked for the AI’s opinions and compared its answers with this “ground truth”. For good measure, we also asked some experienced human viewers to try the same task. When reporting the age of the building, the AI was on average about 4.5 years out. Pretty good. Our human experts were on average 5.9 years wrong – worse than the AI! We saw the same trend for most of the other results: the AI reported the correct type of building 98% of the time, whereas our humans only managed it 91.5% of the time. Our AI reported the right energy source (gas, electricity, etc) 74% of the time while our humans only managed it 38.3% of the time.

It was an exciting discovery and TheSqua.re got there first. We are proud to be one of the most innovative companies in this space, making use of cutting-edge technology.

Must Read: Top Benefits of EcoGrade for Our Customers

Who developed it?

Our talented team of developers and advisors: Prof. Peter Bentley, Dr Soo Ling Lim, Rajat Mathur, Shrey Jindal and Beth Foster.

Where was the work published?

The paper is called: Automated Real-World Sustainability Data Generation from Images of Buildings.

It was published in the IEEE 4th International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) 2014 and was presented by Peter Bentley at the conference in November 2024.

Why publish it as a scientific paper?

Instead of writing a “white paper”, we published our work in a peer-reviewed conference where independent experts ensure our claims are valid before it is accepted. Hence, proving the algorithm of our model is valid. This is why you can trust what we say.

What’s Next?

Using AI in this way is brand new. Nobody has done this before. While this is exciting, it also means there’s still lots to do in order to make it work better. We need to understand how best to teach the latest AIs everything they need to know so that they can correctly interpret images of accommodation in countries all over the world. Prof Peter Bentley’s research lab at UCL is working on this question right now!