Crowdsourcing measures of design quality

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

3 Citations (Scopus)

Abstract

Today there are numerous examples of collaborative online communities effectively creating innovative products (e.g., RepRap, Linux). But the potential of anonymous crowds to also engage in generative design, through the aggregation of many small contributions, is less clear. Although in recent years the “power of the crowd” has been repeatedly demonstrated in areas that range from image labelling to linguistic translation. The application of crowdsourcing in the fields of design research and creative innovation has been much slower to emerge. As a result, although there have been reports of systems and researchers using Internet crowdsourcing to carry out generative design, there are still many gaps in knowledge about the capability and limitations of the technology. For example on commercial crowdsourcing platforms, the relationship between remuneration and the final quality of designs has not been established, so it is unclear how much payment should be offered in order to ensure a particular standard of result. Key to investigating the relationship between the crowd’s remuneration and the value of their innovation is a robust method for quantifying the quality of the designs produced. This paper reports how payment for a design task (a 2D layout problem) was systematically varied and the quality of the output assessed through a separate crowdsourcing process. The work provides some interesting and valuable insight into how Crowdsourcing can be most effectively employed in design tasks.
LanguageEnglish
Title of host publicationASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
Place of PublicationNew York, NY
Number of pages10
Volume1B
DOIs
Publication statusPublished - 2014
Event ASME 2014 International Design and Engineering Technical Conferences & Computers and Information in Engineering Conference - Buffalo-Niagra Convention Center, Buffalo, NY, United States
Duration: 17 Aug 201420 Aug 2014

Conference

Conference ASME 2014 International Design and Engineering Technical Conferences & Computers and Information in Engineering Conference
CountryUnited States
CityBuffalo, NY
Period17/08/1420/08/14

Fingerprint

Innovation
Linguistics
Labeling
Agglomeration
Internet
Linux

Keywords

  • design
  • crowd source
  • online communities

Cite this

Wu, H., Corney, J., & Grant, P. (2014). Crowdsourcing measures of design quality. In ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (Vol. 1B). [DETC2014-34967] New York, NY. https://doi.org/10.1115/DETC2014-34967
Wu, Hao ; Corney, Jonathan ; Grant, Peter. / Crowdsourcing measures of design quality. ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Vol. 1B New York, NY, 2014.
@inproceedings{d74c50e8ec4648d69a493137a489dc93,
title = "Crowdsourcing measures of design quality",
abstract = "Today there are numerous examples of collaborative online communities effectively creating innovative products (e.g., RepRap, Linux). But the potential of anonymous crowds to also engage in generative design, through the aggregation of many small contributions, is less clear. Although in recent years the “power of the crowd” has been repeatedly demonstrated in areas that range from image labelling to linguistic translation. The application of crowdsourcing in the fields of design research and creative innovation has been much slower to emerge. As a result, although there have been reports of systems and researchers using Internet crowdsourcing to carry out generative design, there are still many gaps in knowledge about the capability and limitations of the technology. For example on commercial crowdsourcing platforms, the relationship between remuneration and the final quality of designs has not been established, so it is unclear how much payment should be offered in order to ensure a particular standard of result. Key to investigating the relationship between the crowd’s remuneration and the value of their innovation is a robust method for quantifying the quality of the designs produced. This paper reports how payment for a design task (a 2D layout problem) was systematically varied and the quality of the output assessed through a separate crowdsourcing process. The work provides some interesting and valuable insight into how Crowdsourcing can be most effectively employed in design tasks.",
keywords = "design, crowd source, online communities",
author = "Hao Wu and Jonathan Corney and Peter Grant",
year = "2014",
doi = "10.1115/DETC2014-34967",
language = "English",
isbn = "9780791846292",
volume = "1B",
booktitle = "ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference",

}

Wu, H, Corney, J & Grant, P 2014, Crowdsourcing measures of design quality. in ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. vol. 1B, DETC2014-34967, New York, NY, ASME 2014 International Design and Engineering Technical Conferences & Computers and Information in Engineering Conference, Buffalo, NY, United States, 17/08/14. https://doi.org/10.1115/DETC2014-34967

Crowdsourcing measures of design quality. / Wu, Hao; Corney, Jonathan; Grant, Peter.

ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Vol. 1B New York, NY, 2014. DETC2014-34967.

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - Crowdsourcing measures of design quality

AU - Wu, Hao

AU - Corney, Jonathan

AU - Grant, Peter

PY - 2014

Y1 - 2014

N2 - Today there are numerous examples of collaborative online communities effectively creating innovative products (e.g., RepRap, Linux). But the potential of anonymous crowds to also engage in generative design, through the aggregation of many small contributions, is less clear. Although in recent years the “power of the crowd” has been repeatedly demonstrated in areas that range from image labelling to linguistic translation. The application of crowdsourcing in the fields of design research and creative innovation has been much slower to emerge. As a result, although there have been reports of systems and researchers using Internet crowdsourcing to carry out generative design, there are still many gaps in knowledge about the capability and limitations of the technology. For example on commercial crowdsourcing platforms, the relationship between remuneration and the final quality of designs has not been established, so it is unclear how much payment should be offered in order to ensure a particular standard of result. Key to investigating the relationship between the crowd’s remuneration and the value of their innovation is a robust method for quantifying the quality of the designs produced. This paper reports how payment for a design task (a 2D layout problem) was systematically varied and the quality of the output assessed through a separate crowdsourcing process. The work provides some interesting and valuable insight into how Crowdsourcing can be most effectively employed in design tasks.

AB - Today there are numerous examples of collaborative online communities effectively creating innovative products (e.g., RepRap, Linux). But the potential of anonymous crowds to also engage in generative design, through the aggregation of many small contributions, is less clear. Although in recent years the “power of the crowd” has been repeatedly demonstrated in areas that range from image labelling to linguistic translation. The application of crowdsourcing in the fields of design research and creative innovation has been much slower to emerge. As a result, although there have been reports of systems and researchers using Internet crowdsourcing to carry out generative design, there are still many gaps in knowledge about the capability and limitations of the technology. For example on commercial crowdsourcing platforms, the relationship between remuneration and the final quality of designs has not been established, so it is unclear how much payment should be offered in order to ensure a particular standard of result. Key to investigating the relationship between the crowd’s remuneration and the value of their innovation is a robust method for quantifying the quality of the designs produced. This paper reports how payment for a design task (a 2D layout problem) was systematically varied and the quality of the output assessed through a separate crowdsourcing process. The work provides some interesting and valuable insight into how Crowdsourcing can be most effectively employed in design tasks.

KW - design

KW - crowd source

KW - online communities

UR - http://proceedings.asmedigitalcollection.asme.org/volume.aspx?conferenceid=3431&volumeid=16851

UR - http://www.asmeconferences.org/idetc2014/

U2 - 10.1115/DETC2014-34967

DO - 10.1115/DETC2014-34967

M3 - Conference contribution book

SN - 9780791846292

VL - 1B

BT - ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference

CY - New York, NY

ER -

Wu H, Corney J, Grant P. Crowdsourcing measures of design quality. In ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Vol. 1B. New York, NY. 2014. DETC2014-34967 https://doi.org/10.1115/DETC2014-34967