If a thermostat is essentially an on/off switch for your heating and cooling system, what's there to test? Quite a bit, it turns out, if you're talking about smart thermostats.

CR recently revamped its testing protocol for thermostats, and now the ratings include 14 models of smart thermostats. These are devices that, by virtue of being connected, give you the ability to control and program them using your smartphone. Many of them also talk to other smart-home devices and offer some degree of automation.

The research firm Gartner estimates that 7 million smart thermostats will ship worldwide this year. That might not seem like a lot, but the category only took off in 2011 thanks to Nest. And Gartner expects this number will only grow.

More on Smart Thermostats

The irony is that smart thermostats are complex in order to be simple to use. Nonconnected programmable thermostats were too difficult to use for enough Americans that Energy Star stopped certifying them back in 2009, withdrawing its validation of them as an energy-saving device.

But because of the user interface improvements in smart thermostats, Energy Star began certifying them again this year, making it easier for you to know which models will save you money on your energy bills.

You'll also want to know how well these pricey devices perform. That's where CR comes in, with ratings and reviews on smart thermostats from brands including Carrier, Ecobee, Honeywell, Nest, Schneider Electric, Venstar, and more.

How We Test and Rate Smart Thermostats

Consumer Reports has tested hundreds of thermostats over the years. Any model today should be able to keep your environment close to the set temperature, so we focus our testing on ease of manual control—just like we test regular, nonconnected thermostats—and three things specific to smart thermostats: wireless setup, smart-home features, and automation of heating and cooling.

Smart IQ. Our testing of smart-home features covers things like alerts, app control, and geofencing. To make sure the features work, our engineers drove 3 miles away from our labs with two smartphones to test geofencing. All of these individual features are incorporated into our Smart IQ score, which allows you to see just how “smart” one thermostat is compared with another.

Automation. For automation, we check to see whether the smart thermostats learn habits and routines to adjust temperatures. This feature takes advantage of location tracking, occupancy sensors, and computer algorithms to make adjustments. Because automation is such an important aspect of this product category, we’ve pulled these results into a separate score.

WiFi Setup. Borrowing from our testing parameters for electronics, we tested wireless setup, which includes connecting the thermostats to WiFi, setting up their corresponding smartphone apps, and setting up voice control with Amazon Alexa, Google Assistant, and Apple’s Siri, if those integrations are available. These tests make up a WiFi Setup score for each thermostat.

All of these tests and scores are folded into our overall smart-thermostat scores, giving you a straightforward view of our comprehensive test results.

Smart Thermostats With Different Ideas

Peter Anzalone, the CR test engineer who oversees our thermostat labs, explains that there are three design philosophies apparent in today’s smart thermostats.

“Some smart thermostats are very good at controlling things autonomously without input from the consumer, like those that have geofencing,” he says. “Others have a slick interface geared more for the hands-on type of people who like to program, monitor, and control things themselves. So decide if you're hands-on or hands-off and go from there.”

The first design philosophy is the fully automated approach, like the Nest Learning Thermostat and Nest Thermostat E. These thermostats use sensors and automation to learn your habits so you don’t have to think about your heating and cooling. It’s a good option for consumers who don’t want to be bothered having to program a thermostat and prefer minimal interaction with it.

The second design philosophy is simply that of an internet-connected thermostat with digital controls, like the Carrier Cor thermostat. This thermostat offers digital and remote controls but very little in the way of automation. It’s great for consumers who like programmable thermostats and appreciate added smarts like app and voice control.

The third philosophy is a hybrid of the first two, offering both robust digital controls and a degree of automation. This is best exemplified by the Honeywell Lyric T5 and the Ecobee 3 and Ecobee 4. This kind of thermostat is a good choice for consumers who want some automation but still want to interact with the device.

More to Come

Our new ratings should help you find the best smart thermostat for your home. We’ve also updated our thermostat buying guide to reflect the latest technology and created a step-by-step guide to installing a new device.

And we’re just getting started. Over the next year we’ll be testing more smart thermostats as well as rolling out four other smart-home categories: smart locks, smart smoke and carbon monoxide detectors, home-security cameras, and video doorbells. Smart-home technology is here to stay, and we’re here to help you make sense of it all.