Measuring the positional accuracy of computer assisted surgical tracking systems

Computer Assisted Orthopaedic Surgery (CAOS) technology is constantly evolving with support from a growing number of clinical trials. In contrast, reports of technical accuracy are scarce, with there being no recognized guidelines for independent measurement of the basic static performance of computer assisted systems. To address this problem, a group of surgeons, academics and manufacturers involved in the field of CAOS collaborated with the American Society for Testing and Materials (ASTM) International and drafted a set of standards for measuring and reporting the technical performance of such systems. The aims of this study were to use these proposed guidelines in assessing the positional accuracy of both a commercially available and a novel tracking system. A standardized measurement object model based on the ASTM guidelines was designed and manufactured to provide an array of points in space. Both the Polaris camera with associated active infrared trackers and a novel system that used a small visible-light camera (MicronTracker) were evaluated by measuring distances and single point repeatability. For single point registration the measurements were obtained both manually and with the pointer rigidly clamped to eliminate human movement artifact. The novel system produced unacceptably large distance errors and was not evaluated beyond this stage. The commercial system was precise and its accuracy was well within the expected range. However, when the pointer was held manually, particularly by a novice user, the results were significantly less precise by a factor of almost ten. The ASTM guidelines offer a simple, standardized method for measuring positional accuracy and could be used to enable independent testing of tracking systems. The novel system demonstrated a high level of inaccuracy that made it inappropriate for clinical testing. The commercially available tracking system performed well within expected limits under optimal conditions, but revealed a surprising loss of accuracy when movement artifacts were introduced. Technical validation of systems may give the user community more confidence in CAOS systems as well as highlighting potential sources of point registration error.