Crowdsourcing measures of design quality

Hao Wu, Jonathan Corney, Peter Grant

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

6 Citations (Scopus)


Today there are numerous examples of collaborative online communities effectively creating innovative products (e.g., RepRap, Linux). But the potential of anonymous crowds to also engage in generative design, through the aggregation of many small contributions, is less clear. Although in recent years the “power of the crowd” has been repeatedly demonstrated in areas that range from image labelling to linguistic translation. The application of crowdsourcing in the fields of design research and creative innovation has been much slower to emerge. As a result, although there have been reports of systems and researchers using Internet crowdsourcing to carry out generative design, there are still many gaps in knowledge about the capability and limitations of the technology. For example on commercial crowdsourcing platforms, the relationship between remuneration and the final quality of designs has not been established, so it is unclear how much payment should be offered in order to ensure a particular standard of result. Key to investigating the relationship between the crowd’s remuneration and the value of their innovation is a robust method for quantifying the quality of the designs produced. This paper reports how payment for a design task (a 2D layout problem) was systematically varied and the quality of the output assessed through a separate crowdsourcing process. The work provides some interesting and valuable insight into how Crowdsourcing can be most effectively employed in design tasks.
Original languageEnglish
Title of host publicationASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
Place of PublicationNew York, NY
Number of pages10
Publication statusPublished - 2014
Event ASME 2014 International Design and Engineering Technical Conferences & Computers and Information in Engineering Conference - Buffalo-Niagra Convention Center, Buffalo, NY, United States
Duration: 17 Aug 201420 Aug 2014


Conference ASME 2014 International Design and Engineering Technical Conferences & Computers and Information in Engineering Conference
Country/TerritoryUnited States
CityBuffalo, NY


  • design
  • crowd source
  • online communities


Dive into the research topics of 'Crowdsourcing measures of design quality'. Together they form a unique fingerprint.

Cite this