Performance Characterization of Technology Building Blocks

There are many facets to technology selection. Is the technology at the expected cognition level? How does the performance scale with larger data sets or lot of users? Does the technology perform well most of the time or does it crash or have other bugs? How is the support from the vendor? Last but not the least, what is the cost of using this technology?

Another part of the technology assessment is the quality of the data. If the quality isn’t good or in the right format, we do data cleanup and manipulation. We do a holistic assessment of all the factors that influence the feasibility and outcome of the project.

Pifocal has developed proprietary assessment tools and benchmarks for this service. Our report will show the performance of each vendor and hence help the customer make the right choice of technologies. The process of technology selection is time consuming, involved, and complex. The APIs for other technologies can be tested by running them and seeing if they pass or fail. On the other hand, cognitive APIs must be tested with several appropriately selected datasets. Moreover, they return results in terms of confidence levels. Hence the situation isn’t one of pass/fail work but there are shades of grey and these should be understood properly. Pifocal’s assessment reports and advisory service will help you make the most appropriate technology decisions for your business.

Performance characterization is one of the most important parts in any technology and is often overlooked. Haralick(1,2,3) did pioneering work in computer vision when he introduced performance characterization in the field. This work was further extended with pose estimation and the paper is co-authored by Vinay Vaidya with Haralick(4). Pifocal is now further extending this concept to the field of cognitive technology.

[1] R.M. Haralick, “Performance Assessment of Near Perfect Machines,” Machine Vision and Applications, Vol. 2, No. 1, 1989, pp. 1-16.

[2] R.M. Haralick, “Performance Characterization in Image Analysis: Thinning, A Case in Point”, Pattern Recognition Letters, Vol. 13, January, 1992, pp. 5-12.

[3] Performance characterization in computer vision, in: D. Hogg, R. Boyle (Eds.), Proceedings of the British Machine Vision Conference (BMVC92), Springer-Verlag, Berlin, 1992, pp. 1–8.

[4] R.M. Haralick, Joo, Lee, Zhuang Vaidya, Kim, “Pose Estimation from Corresponding Point Data”, IEEE Trans. on Systems Man and Cybernetics, Vol. 19, No. 6, Nov. / Dec. 1989.