DESIGNING *for humans

The Past Present & Future of User Interface Design Patents

 

Animated graphical user interface for a display screen or portion thereof US D609715 S1

 
Last week I attended the United States Patent & Trademark Office's 7th Design Day.  Design Day 2013 focused on  "Designs in the New Digital Age" - design patent issues related to user interfaces. While UI patents had been touched on in previous years, this was the first time that the day-long meeting placed emphasis on this rapidly growing area, and was clearly motivated by last year's Apple-Samsung case. 

 The attendees  and presenters were predominately lawyers and USPTO examiners, along with a handful of  design experts like myself.  As in previous years, the majority of designers attending were drawn from  Industrial Designers Society of America (IDSA), but with the growth of user interface patents I would expect to see a larger involvement with  interface/interaction designers and organizations in the future.

The opening presentations by the USPTO staff included an update on the written description requirement for design patents by Joel Sincavage, Design Practice Specialist.  Without getting into the details, suffice it to say that the majority of lawyers in the audience were not happy to hear what was being presented and the Q&A turned into what I can only imagine a contentious school board or city council zoning meeting must be like.

This was followed by a much more subdued update regarding the United State's implementation of the Hague Agreement, which will streamline the process of filing design patents across multiple countries.

The subsequent presentations predominately focused on user interface design and comprise my summary.  While the presentations were not structured in a chronological order, there was a clear pattern of discussing the history of UI design patents, current data and future needs:

Past
For context, the patentability of user interfaces has been a topic of discussion for decades, but has only been formalized in recent years and is still evolving.    Cathron Brooks, Supervisory Patent Examiner at the USPTO, described "how it all started" with Xerox applying for design patents on its user interfaces in the 1980s, which catalyzed discussion and debate over what characteristics of user interfaces could be patentable.  

Charles Mauro of MauroNewMedia contextualized the Xerox patents within a timeline, mapping key events in user interface development with important patent events from the 1960s to the present:

 

Charles Mauro's timeline of UI development & patent events

In addition, Masashi Omine, a Deputy Director of the Japan Patent Office (JPO) described the history of interface design patents in his home country, starting from the patenting of liquid crystal display designs (e.g. digital watches) starting in 1986, subsequently extending to graphic images (2006), and more recently, dynamic graphics (2011).  On a related note, China does not currently provide any patent protection for user interface designs, although this may change long-term.

Present
Rob Katz of Banner & Witcoff provided summary statistics on the current state of user interface design patents, which make-up about 2% of all design patent applications.  This translates to roughly 500 UI design patent filings per year with the USPTO, but has been rapidly accelerating. Brooks of the USPTO indicated that "Generated Images" (referring to user interfaces and icons) are the fatest growing segment of design patents.  

It's important to keep in mind that design patents are only one form of intellectual property of protection for user interfaces, which can also be covered by utility patents, trademarks and copyrights.  Mauro suggests that all of these forms of IP protection are needed to cover the entire user experience (UX), but that current processes  are not integrated, requiring a strategic approach. Apple demonstrated such a strategy in their case against Samsung, which included multiple design patents, utility patents and trade dress IP.   Katz  mentioned that it is rare for a design patent to include both aspects of the physical product design and the user interface design, as it is typically more effective to patent each separately. 

 

Charles Mauro's depiction of the multiple forms of UX IP protection

 While user interface patents are growing in prominence and value, there are constraints in the design patent application process.  For example Brooks discussed some of the basic requirements for user interface design patents, which include referencing a physical object or system such as a computer or display screen.  Dynamic interactions such as animations or transitions can be submitted as a series of static images like a "flip book" or storyboard, but only those images presented are covered by the patent.

Future
The takeaways from Design Day 2013 were clear - user interface design patents are more important now than ever, and there is a need to evolve the processes around protecting them.  In the short-term this includes increasing staffing at the USPTO to support the growing number of UI design patent applications, and to continue the dialogue between stakeholders at events like Design Day.  I also strongly recommend that user interface designers improve their education and processes around patents.  

Looking further ahead I would expect the USPTO processes to catch-up (or at least narrow the gap) with the design and technology  it is intended to protect.  This may mean more suitable forms for documenting and describing interactive designs as well as a more cohesive approach to combining related forms of IP such as design and utility patents as appropriate.

Finally, consider what David Kappos, former director of the USPTO, wrote last week in the National Law Journal in a piece entitled Design: The New Frontier of Intellectual Property.  Kappos, perhaps anticipating the discussions that would take place at Design Day 2013, admonished:

"But for the breakthrough ­innovators of the 21st century, design has moved onto a much larger stage. It is where high function meets high style. And the traditional disciplines of IP — patents, trademarks and copyrights — are no longer ends unto themselves but are now viewed as component parts of a larger whole."

FieldCREW Concept on Windows Phone

In 2008 Bresslergroup presented FieldCREW - a concept tablet and software system empowering user researchers and designers conducting field research.  It included features for real-time video capture and tagging of observed activities, allowing a single researcher to have the observation capabilities of a team, and a team of observers to synchronize their observations as though they were a single researcher.  FieldCREW recevied widespread coverage on sites including SlashGear, the Awesomer, and Core77.

Fast forward five years, and to the Core77/Windows Phone App to the Future Design Challenge. Now we have reconceptualized FieldCREW as an app, providing the same core features as the original concept, in a more realizable and usable format (but, still a concept).

• 
• 
• 

The Bresslergroup app team, led by myself and interaction designer Matt Thomas, created the information architecture and visual design  for the app version based on the Windows Phone guidelines.  

Watch this the video for more information and screen examples for the Windows 8 phone concept version of FieldCREW:

  

 

Patent Skills for User Experience Design

The Design Story of the Year

The Apple/Samsung patent case was arguably the most important design event of 2012.  Not just because it gave a measureable value to design patents and the role of intellectual property in user experience design, but because of its visibility and potential impact on design processes.  

Over the last couple of weeks, two important thought pieces were published that bring attention to this impact. 

In his excellent blog, Foss Patents, Florian Mueller discusses the Apple '915 "pinch-to-zoom" patent.  Mueller gets to the heart of the matter when he considers non-infringing alternatives (i.e. Samsung's) to this interaction: 

"there's no question that one can work around that claim limitation, like any claim limitation of any given patent, in a legal sense, and there's also no question that the '915 patent doesn't cover all scrolling and all pinching operations, but the question is what price one has to pay for non-infringement in terms of a degradation of the user experience. Is it intuitive?"

Mueller continues:

"But the question is whether the pinch-to-zoom gesture that Apple has made so popular that it's commonly expected, and instinctively performed, by users can be implemented without infringing the '915 patent or adversely affecting the user experience."

Mueller is implying (perhaps not intentionally) that effective interaction design, requires  awareness of what interactions are infringing and the consequences of designing around them.  Taking this a step further, we might consider whether the value of infringing, or rather licensing to avoid infringement, provides a greater user experience than developing a non-infringing alternative.  I posit that it is extremely rare for interaction designers to think in these terms when deciding a design direction.

Charles Mauro gets much more direct and specific about the implications of intellectual property on interaction design in his comprehensive analysis of the Apple v. Samsung case.  Mauro starts out by providing a clear review of the case and the issues at stake, but it is his synthesis and conlusions that are most relevant here.  Mauro argues that it was not the individual design patent, utility patent or trademarks that were at stake, but the collection of Apple intellectual property which collectively define the total user experience:

"The asserted Apple patents covered the look of the iPhone, the style of the screen-based interface, design of the icons and related screens, and some of the essential gesture-based interaction behaviors of the iPhone product. The sum of these patents covered essentially the iPhone you may or may not have in your hand from the standpoint of what you cognitively experience when you use the device. In retrospect this was a brilliant strategy because it gave Apple the opportunity to secure (conceptually) protections well beyond those that accrue to any single patent related to iPhone technology.  This was likely an essential legal strategy of Apple since it is the Total User Experience (TUX) of the iPhone that drives commercial success, not any individual feature or function as presented in a single patent asserted."

Consequently, Mauro advises, intellectual property development needs to focus broadly on the overall user experience, not narrowly on individual design elements and functions:

" IP practitioners should make use of all available forms of IP protection including design patents, utility patents, copyright, trademark and in litigation, trade dress, to create an interlocking set of protections that cover the look, feel and function of your products and services"

 Again, I would argue that when designers are creating a product design strategy, patents are not at top of mind.

Will Design Adapt?

After the Apple v. Samsung case, everyone seems to get the importance of interaction design to intellectual property law.  But are we seeing the importance of intellectual property law in the design process?  Both Mueller and Mauro understand the symbiosis, even if the design community generally does not.  

Despite the high importance of patents to the design process, the area still receives relatively little attention among the design community.  For example, the program for the industry-leading Interaction13 conference does not include a single presentation on the topic of patents or intellectual property, despite the huge visibility and importance of the Apple v. Samsung case.

For those designers, who are interested in designing effectively with patents, here are three key steps to getting started

1. Gain an understanding of the fundamentals of patents.  There are various resources for this, and I have yet to find a single one that I strongly recommend, so please provide your suggestions in the comments.  Keep in mind this is not a one-time read and learn activity. Patents are a dynamic area and it is important to stay informed in an ongoing manner.   A couple of recent recommendations include a section run in the NYTimes earlier this year (recommended for interaction designers) and a series on Core 77 about design patents (recommended for industrial designers).    Also, PatentlyO is a blog/feed for daily updates on patent news - not necessarily always relevant for a designer, but an effective way to stay in the loop.
2. Read the existing patents in your area of design and track new patent applications going forward. I personally like Arch Patent which is a recent newcomer and has some nice search and filtering tools, but of course there's always Google's patent search.
3. Integrate patent review as part of your design process.  In many organizations this is done after the creative process, to check for potential infringment.  I advocate also reviewing patents before and during the creative process so that designers understand the constraints and opportunities available.  Just as designers advocate understanding user needs and business goals in order to design effectively, they should also embrace knowing the intellectual property space.

 

Update 17 Dec 2012 - Matthew Smith has posted an overview of design patents that can serve as a reference and introduction to the legal details (courtesy of PatentlyO).

 

Bureau of Ergonomics - Holiday Edition

The latest Bureau of Ergonomics discusses the frustrations of impossible packaging (with a pretty cool illustration), and the future of voice interfaces.  In case you missed it, catch-up on the the initial BoE column.

Apple's New Ergonomic Posture

Although not as fun as Samsung's current advertising campaign, Apple's latest television spots are notable for their focus on ergonomics.  While Apple has always been know for marketing their products based on simplicity, ease of use and aesthetics, their newest TV campaign puts a direct emphasis on the physical ergonomics of their products.  For example the "Thumb" commercial compares the range of  thumb reach to the screen layout of the iPhone 5:

  

Another one of their ads, "Ears" touches on the anthropometrics of the outer ear (although certainly not using those technical terms).  Here's the long-form version, not typically seen on TV, with Jony Ive discussing the 3D scanning of ears to determine best fit:

 

Why is Apple marketing a new emphasis on physical ergonomics?   Perhaps with the growing intersection of digital interaction and physical design, an area that Apple has pioneered via gestural interactions of the iPhone and iPad, consumers are  looking beyond the flat-world of the digital display to the more holistic three-dimensional aspects of the user experience. 

As an ergonomist it's interesting to see how potential ergonomic benefits are defined and communicated to the consumer public.  The thumb advertisment is intentionally simplistic and non-technical, almost poetic.  The long-form ears video is certainly more interesting and explanatory, and Jony Ive admits, albeit subtley, the challenges in designing a one-size-fits-all product:

"Making one headphone to fit everybody's ears would be like trying to make one pair of shoes to fit everbodys feet - I mean it's impossible.  But that's exactly what we've tried to do with the new EarPods." (emphasis added)

It's important to note that there's a difference between marketing ergonomics and designing ergonomics (often a huge difference).  For instance their are some concerns about the just announced iPad Mini, including the fesability of one-handed grasping  (see A Mini Tablet for a Maxi Hand).

It will be interesting to see whether Apple continues this emphasis in its marketing and whether competitors start doing likewise.

Update Dec 1 - I learned of Dustin Curtis' examination of thumb reach for the 3.5" vs 4" versions of the iPhone.  Although seemingly based on his own thumb dimensions, Curtis extrapolates out some interesting insights on the ergonomics of the iPhone 5:

"Four inches is only now barely acceptable on iPhone 5 because:

• iPhone 5’s huge reduction in weight makes it easier to hold while contorting your hand to touch the hard-to-reach areas of the screen.
• The screen’s width remains narrow and only grows vertically, meaning it’s still easy to reach the entire width of the device;
• The device is 20% thinner, which allows your hand to wrap around more fully and to gain slightly more reach; and
• iOS’s tab bars are anchored to the bottom of the screen, where your thumb more naturally rests, so it remains easy to change app sections (contrast this with Android’s tab bars, which are usually located at the top of the screen, and sometimes out of reach)."

For User Interfaces, Cracks in the Patent System

The New York Times has published a very interesting article and interactive section on the trends and issues in software patents, particularly relevant to mobile user interfaces.  The main article, The Patent, Used as a Sword, posits that the current patent system, which was founded around mechanical devices, is less than effective for the more conceptual and ambiguous world of software patents.  Not surprisingly, it focuses on recent Apple-related cases,  including a chart, mapping out the players in recent smartphone patent suits.

Of particular interest is the paired interactive content including a video and comparative examples of some well-known Apple patents with associated prior art (pictured above).  The Times asks the reader "if you were on a jury, what would you decide?"

Also of interest to UI patent followers, the upcoming NYTech presentation Apple vs Samsung: What the Case Means to Software Development and UX Design, which will be moderated by my colleague Charles Mauro.