• Provided a simple bar chart and scoring solution
  • Now that the site was going to show comparisons between a specific business and its peers, we needed a compelling way to present this information. To get started, we explored a variety of different design options for how best to show comparison data. Next, we usability tested our ideas and moved forward with the design that users liked best. Our final solution combined a simple bar chart with a visual quartile score, in a format that was easy for scanning.

• Created a clean, fresh, and consistent visual design
  • We all know that it is important for information on a site to not only be useful, but to also be presented in a clear, visually appealing way. To help the Trends site appeal to its audience and convey the desired tone, we created a modern, clean look and feel for the site that also followed Intuit’s brand guidelines. We also introduced a consistent color scheme to use for “Me” and “Peers” across the site to help users differentiate between the two types of data.

You can check out our redesign for yourself at: https://workplace.intuit.com/db/bejqb2kpn.
We’d love to hear what you think!

2. Create a goal – get commitment – provide feedback
Consider structuring the DR program so that participants get a specific difficult goal for participation, commit to the goal, and then get feedback on their goal. This type of structure has proven repeatedly to be one of the strongest approaches for motivating energy behavior change. In one study, researchers gave households a difficult goal (20% energy reduction), easy goal (2% reduction), or no goal for energy use. All groups (including the no goal control) were then given information on which appliances used the most energy. The goal was also combined with feedback or not. Households who received a difficult goal + feedback conserved the most (15.1%) and were the only group to significantly differ from the control. Participants with the easy goal did not differ in behavior from the control at all. To make an even stronger program, consider an extra reward if the goal is reached.

3. Provide frequent feedback
The more continuous the feedback, the more effective the intervention. In a seminal study conducted over 30 years ago in 1979, households were given continuous feedback over a period of 11 months about monetary costs of electricity use by means of a monitor displaying electricity use cents per hour. On average, households that had a monitor installed reduced electricity by 12%. Although hourly, daily, weekly, and monthly feedback all create savings effects, the more frequent the feedback, the more effective it is. Consider creative ways to deliver that feedback via web portals, in home devices, smart phones or on SMS (intermittently).

4. Emphasize choice and control
One study considered people’s resistance to installing automatic day/night thermostats. Once the thermostat was redesigned to allow residents to override the system temporarily, the thermostat was much more attractive to residents – even though in actual use most people never overrode them. Similarly, a DR program should emphasize choice and control – people can opt into the program and still have full control over their consumption.

5. Tap into the power of the group
One energy conservation program that had a lot of success enrolled people in groups where they discussed and compared conservation behavior with their social group over a long term basis. Similarly, virtual networks of known groups can be set up to motivate participation in DR programs – for example by tapping into existing social networks of friends on Facebook to encourage participation.
In addition, consider a structure that offers additional savings if everyone in a group or neighborhood participates and reaches a set goal (see point 1 above). One study indicated that an incentive that offered on an individual and group level – in this case for all residents of one apartment building — was more effective than solely an individual incentive.

6. Frame program benefits as avoiding loss rather than emphasizing gain
The amount of joy that someone experiences when winning $100 is not equal to the consternation suffered when losing the same amount. Most people are more willing to take risks to avoid or minimize a loss than to increase their fortune. So, focus on showing residents how much money they are losing every month by not enrolling in demand response. Once the loss is obvious, people will take action.

7. Integrate complex information
When calculating energy savings, people usually can’t take into account all the elements such as rising fuel costs, the real long terms benefit, etc… So, do the math for them! Give price information that shows the full savings, presented as avoidance of a negative consequence of non-action (see point above). Use the actual data you already know about the consumer’s energy usage to make the information actionable and real.

8. Present information using vivid personal stories and videos
Statistical data summaries and impersonal information are less effective than case studies and colorful stories for motivating participation. For example, imagine that you are considering a new car and are choosing between a Volvo and a Saab. Consumer Reports informs you that the consensus of its studies is that the Volvo has a better repair record. That evening, you go to a party and run into an acquaintance who tells you a horrific story about a Volvo. Although the Consumer Reports article is based on hundred of repair records and your friend’s story is just one additional data point, most people will be swayed by their friend not to buy that car.
When communicating the benefits of a direct response program, demonstrate benefits with concrete stories about real people who save more energy than average but are “just like you. To be even more effective, present the content in videos. Numerous studies have also shown that videos of people modeling the desired actions are more effective in getting people to change their behavior than written information or lectures.

9. Use a foot in the door strategy
Individuals who agree to a small initial task are much more likely to agree to a larger request. So, instead of asking people to enroll in the full DR program immediately, first ask people to participate in a small act, such as filling out a survey, and then later ask them to consider signing up for demand response as a follow up to the first request. For example, one representative study showed that the percentage of people agreeing to an unattractive sign being put on their front lawn encouraging people to drive carefully increased dramatically (from 17% to 55%) if they had first been given the opportunity to sign a petition favoring safe driving.

10. Communicate trust
One key differentiator for successful energy programs is successful marketing to get people to even consider trying it out. We’ve found in our research that people inherently don’t trust their utility so partner with a local organization people do trust to market your program. In a marketing experiment conducted in Minnesota, a county government contracted with a private company to install energy saving equipment in homes in exchange for payment of a percentage of the value of the energy saved. To market the program, households received one of three types of letters: one letter was sent on company letterhead with no mention of cooperation with the county, one letter went out on company letterhead and mentioned the county’s role, and the third went out on county letterhead and was signed by the County Board of Commissioners. The source of information had a profound effect on consumer response – request for energy audits came from 6%, 11% and 26% respectively of households receiving the three types of letters.

Next Steps

We’re continuing to do more research in this area and will publish more insight expanding into some areas mentioned above. In the meantime, here’s a partial list of references…happy reading!

• Abrahamsen, W., Steg, L., Vlek, C., & Rothengatter, T. (2005). A review of intervention studies aimed at household energy conservation. Journal of Environmental Psychology, 25, 273-291.
• Krantz, D. H., & Kunreuther, H. C. (2007). Goals and plans in decision making. Judgment and Decision Making, 2(3), 137-168.
• Geller, E. S., (1992) It takes more than information to save energy. American Psychologist, 814-815.
• Geller, E. S., Winett, R. A., & Everett, P. B. (1982). Preserving the environment: New strategies for behavior change. Elmsford, NY: Pergamon Press.
• Lutzenhiser, S. et al (2009) Beyond the Price Effect in Time-of-Use Programs: Results from a Municipal Utility Pilot, 2007-2008. Presented at the International Energy Program Evaluation Conference, Portland, OR, August 12-14, 2009. http://drrc.lbl.gov/pubs/lbnl-2750e.pdf
• McKenzie-Mohr, D. and Smith, W. (1999) Fostering Sustainable Behavior. Gabriola Island, B.C., Canada: New Society Publishers.
• Swim, Janet et al. (2009) Psychology and global climate change: addressing a multi-faceted  phenomenon and set of challenges, A report by the American Psychological Association Task Force on the Interface between psychology and global climate change. http://www.apa.org/science/about/publications/climate-change.aspx
• Winett, R. A., Hatcher, J. W., Fort, T. R., Leckliter, I. N., Love, S. Q., Riley, A. W., et al. (1982). The effects of videotape modeling and daily feedback on residential electricity conservation, home temperature and humidity, perceived comfort, and clothing worn: Winter and summer. Journal of Applied Behavior Analysis, 15(3), 381-402.
• Winett, R.A. and Geller, E.S., (1981) Comment on “Psychological research and energy policy”. American Psychologist, (425-426).
• Yates, S. and Aronson, E., (1983) A social psychological perspective on energy conservation in residential buildings. American Psychologist, (435-444).
• Stern, P. C., Aronson, E., Darley, J. M., Hill, D. H., Hirst, E., & Kempton, W. et al. (1986). The effectiveness of incentives for residential energy conservation. Evaluation Review, 10(2),
  147-176.

For related posts about designing for the smart grid, check out:

• Tips For Improving Greenbox’s Energy Portal
• Watts All the Buzz  About Smart Grid Energy?

1) Simplify, simplify, simplify!

My foremost suggestion is simplicity. At first glance, the Greenbox interface looks very busy. There are many controls and buttons, and it tries to accomplish too much on a single page. Do users really need to be able to adjust their thermostat controls right on the main landing page, and also view their overview and data details at the same time? Instead, the home page could show only the most important information at first glance, and use color to further highlight and differentiate content on the page. The user can always drill-in to inside pages to access the less urgent information or access info on mouseover.

2) Engage users immediately with compelling content and proactive recommendations

My first response to the Greenbox dashboard is, “So what?”  I’m missing context to understand the significance of my data. For example, is my overall usage good or bad? Am I using more or less energy than I did last month? Am I on track so far this month? Also, the portal doesn’t offer any suggestions for changing my behavior so it’s unclear what my next step should be.

To engage its users, Greenbox should focus on delivering insightful content that immediately provides context and motivation. Delight users by anticipating their questions and displaying answers at-a-glance. Provide simple charts and visuals on the dashboard to compare energy usage to different time periods and to others in the community so that people understand why this information matters.

3) Adhere to basic design principles

In addition to overall experience enhancements, here are some basic user interface guidelines to help GreenBox or any other site be more usable:

• Use clear titles Is the bar chart below showing data for all energy resource usage, just the electricity usage, or only heating and cooling (because that’s what is selected in green)? It’s hard to know for sure without a clear title for the chart.

• Use visual cues to help users appropriately group related itemsThe content relationships on GreenBox are a little hard to parse.  For example, in the screenshot below, the “Heating/Cooling” button selected on the left side of the dashboard, actually updates the content shown on the right side, which is not immediately obvious.

Instead, GreenBox could make these two items more visually related to connect them. For example, put the same  green border around the box on the right and give both elements the same background color (see update below). Then, I would be much more likely to quickly understand their relationship.

Updated design with green box:

• Visually identify hyperlinks & interactive featuresTo help users understand what’s clickable, it’s important to differentiate hyperlinks and interactive features from other static elements on the page. For example, on the GreenBox site buttons and links currently use the same dark text color as other static elements, which makes them difficult to identify.

A Minimal Redesign Idea

If we take the recommendations above and do a very minimal redesign of Greenbox limiting ourselves to only minor changes to content and layout, we already begin to see large improvements in terms of usability.

For example, in this minimal redesign, the relationships between content is clearer due to stronger visual cues and use of titles, the interactive elements more readily “pop” from the page, and inclusion of arrow icons begin to tell an interesting story about the user’s energy consumption.

But we’re not done yet…

Of course, it would take GreenBox more than just a simple redesign with no change in functionality to address all of our recommendations.  As a next step, I think Greenbox would benefit to take a more critical look at its overall structure and content to ensure they lend themselves to a good user experience – in particular thinking about the best items to put on the home page and trying to answer the question “So what?”

Will we see a new and improved Greenbox site in the near future?  We hope so.  In the meantime, let’s learn both from the things that Greenbox does well and from the areas that Greenbox falls short so that we can continue to focus our energy on creating intuitive, meaningful, and persuasive user experiences.

Related post: Watts all the buzz about smart grid energy?

But, do you mean retention or forced retention? For example, I can artificially increase my retention numbers by locking new users into a long contract, and that’s going to keep them with me for awhile. Once that contract’s up, they are free to move wherever they like, and then I need to acquire a customer to replace them. And, if my contract is longer than my competitors’, it can scare off new users.

Also, the retention metric is easy to affect with switching barriers, which may increase the number of months I have a customer while making them less happy. Of course, if those switching barriers are removed for any reason – for example, cell phone number portability – I can lose my hold over long time customers.

While retention can be an indicator of happy customers, increasing retention by any means necessary doesn’t necessarily make your customers happier.

Revenue

Revenue’s another metric that seems like it would point to happy customers. Increased revenue means people are spending more, which means they like your service!

There are all sorts of ways I can increase my revenue without making my customers happier. For example, I can rope them into paying for things they didn’t ask for or use deceptive strategies to get them to sign up for expensive subscriptions. This can work in the short term, but it’s likely to make some customers very unhappy, and maybe make them ex-customers in the long run.

Revenue is also tricky to judge for free or ad-supported products. Again, you can boost ad revenue on a site simply by piling more ads onto a page, but that doesn’t necessarily enhance your users’ experience or happiness.

While increased revenue may indicate that people are spending more because they find your product more appealing, it can also be caused by sacrificing long term revenue for short term gains.

NPS – Net Promoter Score

The net promoter score is a measure of how many of your users would recommend your product to a friend. It’s actually a pretty good measure of customer happiness, but the problem is that it can be tricky to gauge accurately. It generally needs to be obtained through surveys and customer contact rather than simple analytics, so it suffers from relying on self-reported data and small sample sizes. Also, it tends to be skewed in favor of the type of people who answer surveys and polls, which may or may not be representative of your customer base.

While NPS may be the best indicator of customer happiness, it can be difficult to collect accurately. Unless your sample size is quite large, the variability from week to week can make it tough to see smaller changes that may warn of a coming trend.

Conversion to Paying

For products using the freemium or browsing model, this can be a useful metric, since it lets you know that people like your free offering enough to pay for it. However, it can take awhile to collect the data after you make a change to your product because you have to wait for enough new users to convert to payers.

Also, it doesn’t work well on ad-supported products or products that require payment upfront.

Most importantly, it doesn’t let you know how happy your paying customers are, since they’ve already converted.

Conversion to Paying can be useful, but it is limited to freemium or browsing models, and it tends to skew toward measuring the free part of the product rather than the paid product.

Engagement

Engagement is an interesting metric to study, since it tells me how soon and often users are electing to come back to interact with my product and how long they’re spending. This can definitely be one of the indicators of customer happiness for ecommerce, social networking, or gaming products that want to maximize the amount of time spent by each user. However, increasing engagement for a utility product like processing payroll or managing personal information might actually be an indicator that users are being forced to do more work than they’d like.

Also, engagement is one of the easiest metrics to manipulate in the short run. One time efforts, like marketing campaigns, special offers, or prize giveaways can temporarily increase engagement, but unless they’re sustainable and cost effective, they’re not going to contribute to the long term happiness of your customers.

For example, one company I worked with tried inflating their engagement numbers by offering prizes for coming back repeatedly for the first few days. While this did get people to return after their first visit, it didn’t actually have any effect on long term user happiness or adoption rates.

Engagement can be one factor in determining customer happiness, but this may not apply if you don’t have an entertainment or shopping product. Also, make sure your engagement numbers are being driven by actual customer enjoyment of your product and not by artificial tricks.

Registration

While registration can be the fastest metric to see changes in, it’s basically worthless for figuring out how happy your users are, since they’re not interacting with the product until after they’ve registered. The obvious exception is products with delayed (i.e. lazy) registration, in which case it can act like a lower barrier-to-entry version of Conversion to Paying. When you allow users to use your product for awhile before committing, an increase in registration can mean that users are finding your product compelling enough to take the next step and register.

Registration is only an indicator of happy customers when it’s lazy, and even then it’s only a piece of the puzzle, albeit an important one.

Customer Service Contacts

You’d think that decreasing the number of calls and emails to your customer service team would give you a pretty good idea of how happy your customers are. Unfortunately, this one can be manipulated aggressively by nasty tactics like making it harder to get to a representative or find a phone number. A sudden decrease in the number of support calls might mean that people are having far fewer problems. Or, it might mean that people have given up trying to contact you and gone somewhere else.

Decreased Customer Service Contacts may be caused by happier customers, but that’s not always the case.

So which is it?

While all of these metrics can be extremely important to your business, no single one can tell you if you are making your customers happy. However, looking at trends in all of them can certainly help you determine whether a recent change to your product has made your customers happier.

For example, imagine that you introduce a new element to your social networking site that reminds users of their friends’ birthdays and then helps them choose and buy the perfect gifts. Before you release the feature, you decide that it is likely to positively affect:

• Engagement – every time you send a reminder of a birthday, it gives the user a reason to come back to the product and reengage.
• Revenue – assuming you are taking a cut of the gift revenue, you should see an increase when people find and buy presents.
• Conversion to Paying – you’re giving your users a new reason to spend money.
• (Lazy) Registration – if you only allow registered users to take advantage of the new feature, this can give people a reason to register.
• Retention – you’re giving users a reason to stay with you and keep coming back year after year, since people keep having birthdays.

Once the feature is released, you look at those numbers and see a statistically significant positive movement in all or most of those metrics. As long as the numbers aren’t being inflated by tricks or unsustainable methods (for example, you’re selling the gifts at a huge loss, or you’re giving people extra birthdays), you can assume that your customers are being made happy by your new feature and that the feature will have a positive impact on your business.

Of course, while you’re looking at all of your numbers and metrics and analysis, some good old fashioned customer outreach, where you actually get out and talk directly with users, can also do wonders for your understanding of WHY they’re feeling the way they’re feeling. But that’s another post.

Interested? You should follow me on Twitter.

For more information on the user experience, check out:

• 6 Reasons Users Hate Your New Feature
• Is Continuous Deployment Good For Users?
• A Faster Horse – When Not to Listen to Users
• Improving the ROI for Your Customer Feedback
• 5 Things People Get Wrong When Talking to Users

Your Product Interface Is a Giant Mess

Remember the old Saturday Night Live spoof commercial that advertised, “It’s a floor wax! It’s a dessert topping!”? It’s not as funny when it’s true. Products cannot do everything, and when they try, they end up with interfaces far too complicated for the average user to navigate.

I see this happen all the time, especially with startups looking for a way to make their product appeal to more people. Instead of improving their core product and adding features that enhance that experience, they add unrelated feature after unrelated feature, often stolen directly from more successful companies with larger user bases. Their goal is to find something that makes them blow up huge, but they just end up with an overly complicated product that tries to do too many things and doesn’t do any of them well.

It’s not just startups that suffer from this. Products that have been around for many years often get bogged down with feature after feature, all of which have to be supported because some fraction of the user base still uses it. These products then become vulnerable to new challengers with more focused, easy to use interfaces and smaller feature sets.

Of course, there are times when companies have to take their products in a new direction. For example, Flickr started as a set of tools for an MMORPG called Game Neverending. The game has ended, but Flickr lives on as an entirely different business.  PayPal began as a way to make PDA to PDA mobile payments, but that feature got killed years ago when they realized that web payments were a far better business model.

When you do find that killer feature that’s going to change your whole business model, commit to it and make it a serious focus rather than burying it under dozens of less popular features. Don’t try to be all things to all people, or you will end up with nothing but a giant, unusable mess.

Before adding a new feature, ask yourself, “Does this enhance my current product experience or just add to an already confusing and cluttered interface?” And, if it doesn’t fit with your current product offering but you still want to do it, ask, “Am I prepared to cut other features to make this part of my core offering and simplify my experience?”

You Didn’t Build What They Asked For

Let’s face it, sometimes your priorities are different from your users’. For whatever reason, be it a new business partnership, a need for a new revenue stream, or the desire to attract a different group of users, sometimes you’re going to build something that your current users don’t want and didn’t ask for.

This isn’t always a bad thing. For example, something that annoys your current non-paying users but attracts a whole slew of new, paying users is worth a few nasty emails to your customer service department. Just make sure that the new feature is really going to do what you think it will. It sucks to piss off your current, paying customers to build a feature that never really fulfills its initial promise. Trust me on this.

Before building a feature that potentially has more benefits to your company than your current users, ask yourself, “Am I prepared to deal with the fact that this is going to annoy some of my customers, and what is the real likelihood that I will get more out of this than I will lose?”

You Built EXACTLY What They Asked For

I know. It doesn’t seem fair. They’re angry if you don’t do what they want, and they’re angry if you do what they want.

The truth is that users will often ask you for a solution when it would really be more helpful to tell you that they have a problem. I’ve written more extensively about when you shouldn’t be listening to your users, but the upshot is that users aren’t great predictors of which brand new features will be big hits. Sometimes users will tell you that they want a toaster in their car, when what they really mean is that they don’t have time to make breakfast in the morning.

Before building a new feature that your customers are demanding, ask yourself, “What known user problems is this solving, and is this the best way to solve it for everybody?”

The Feature’s Not Finished

Now, I’m all for building the minimum viable product, getting it out in front of users, and then iterating on it to improve it, but some features just aren’t ready for prime time. By launching a half baked feature without key functionality, you’re running the risk of turning a lot of people off on the idea before they ever get a chance to really try it out.

Remember, your customers aren’t in the conference room with you when you come up with your grand vision. They don’t know where you’re going with this neat new idea. They’re judging the feature based on their first experience with it. Make sure that the first version is at least usable and hopefully that it’s far enough along that users can see the same promise that you do.

Also, good enough to ship doesn’t necessarily mean good enough to remain in your product long term. A big part of shipping early is continuing to improve the feature once it’s been out for awhile. One company I worked with had the tendency to ship early versions of features and then let them just sit there gathering dust, rather than iterating on them until they were truly high quality. What they ended up with was an enormous product that all seemed about half finished and a lot of unhappy customers who didn’t believe features would ever improve past version 1.0.

Before shipping a new feature, ask, “Is this good enough that users will get why they should care about it? And, if they do care about it, am I committed to improving it?”

The Feature’s Not Finishable

At many of the companies I’ve worked with, features have tended to evolve before they even get built. What generally happens is this: you have an idea based on something you’ve heard from users; that idea gets brainstormed and grows based on internal input; UX and visual designers spec out the whole idea, often expanding on the original idea; then engineering gets involved and gives a time estimate of how long the feature will take to build; finally the whole thing gets cut back by about 80% based on the estimates.

Unfortunately, the 20% you end up implementing may not solve the original problem. That means, when you finally announce your great new feature, users who originally asked to have that particular problem solved are justifiably upset.

Before drastically cutting your new feature back, ask yourself, “Does this still solve the original problem I was trying to solve?” If it doesn’t, ask, “Can this problem be solved with a reasonable level of investment?” There’s no shame in answering no to either or both of these questions, as long as you decide not to go forward with the new feature.

The Secret to Making Everybody Love Everything You Make

I’m joking. There’s no secret. The truth is that it’s almost certainly impossible. But by asking yourself the right questions during your feature development phase, you can dramatically cut back on time spent creating things your users hate.

And never forget, when you do build something they hate, acknowledge it, apologize for it, and fix it. It will go a long way toward making your users happy again, and it might even get them to like that neat new feature you just shipped.

Interested? You should follow me on Twitter.

For more information on the user experience, check out:

• Is Continuous Deployment Good For Users?
• A Faster Horse – When Not to Listen to Users
• Improving the ROI for Your Customer Feedback
• 5 Things People Get Wrong When Talking to Users
• A/B and Qualitative User Testing