CreatiCampusAn Esquine scholarly journal

Two Numbers, One Record: A Study Protocol for Testing Whether Separate Reporting of Planned and Extra Output Makes IMF Records Comparable as Creative-Campus Data

Untung Rahardja, Qurotul Aini, Nuke Puji Lestari Santoso, Marviola Hardini, Fitra Putri Oganda

· CreatiCampus Working Paper Series · Vol. 1, Issue 1 (2026)

Abstract

Indicator Measurement Factor Analysis (IMF) records creative-campus projects as missions with planned outputs, extra outputs and an achievement percentage. A concept analysis earlier in this series proposed (P4) that reporting planned achievement and extra output as two separate numbers makes IMF records comparable across missions and cohorts and therefore usable as campus-level creative-economy data. This paper is a conceptual and study-protocol paper that prepares the test of that proposition; it reports no empirical results. We show that a single combined percentage admits at least three defensible readings once a mission produces extra output, and model a reported figure as a record component plus a reader component with a judgement part and a convention part, the latter removed by two-number reporting. We state four hypotheses with pre-specified decision rules, and give a fully crossed rater protocol in which independent, trained raters recompute figures from the same de-identified records under a single-percentage condition, a two-number condition and a sensitivity condition with a fully specified single formula. The analysis plan uses the absolute-agreement intraclass correlation ICC(A,1) and Krippendorff's alpha with paired bootstrap comparisons, limits of agreement, a mixed model for moderation by the share of extra output, robustness checks and a planning simulation for sample size (80 records, four raters). A worked example applying the protocol to six missions shows the computations end to end and how the convention part alone can lower agreement on a single figure without any rater error. The protocol is designed so that support for the two-number convention, support for a precisely specified single rule, or a negative result would each be informative for campuses that report creative-economy outputs.

Download full text (PDF)

Check citations on Google Scholar

Cite as: Untung Rahardja; Qurotul Aini; Nuke Puji Lestari Santoso; Marviola Hardini; Fitra Putri Oganda (2026). Two Numbers, One Record: A Study Protocol for Testing Whether Separate Reporting of Planned and Extra Output Makes IMF Records Comparable as Creative-Campus Data. CreatiCampus Working Paper Series, Vol. 1, Issue 1. Esquine CreatiResearch Institute. https://creaticampus.esquine.com/papers/two-numbers-one-record-imf-inter-rater-agreement-protocol/

References

  1. U. Rahardja, L. Sulistyo, D. Safarina, M. Rapidan Kusuma, N. Silawati, and Z. Nanle, "HibahQu Education Monitoring Platform Based on Human-Centric Orange Technology Laravel 12 Vue.js," ADI Bisnis Digit. Interdisiplin J., vol. 6, no. 2, pp. 203–218, Dec. 2025, doi: 10.34306/abdi.v6i2.1348
  2. I. P. Gustiah, N. Lutfiani, M. R. Kusuma, E. P. Lestari, and T. Green, "Grant Submission Monitoring System Based on Orange Technology Using Laravel 12 and Vue.js," J. Orange Technol., vol. 1, no. 2, pp. 75–90, Apr. 2025, doi: 10.68012/jot.v1i2.24
  3. S. D. Sugiyanti, R. Widayanti, M. B. Ulum, G. Firmansyah, and A. H. Azizah, "Design Dashboard Monitoring Teacher Performance Assessment at Cinta Kasih Tzu Chi High School," IAIC Trans. Sustain. Digit. Innov., vol. 4, no. 1, pp. 46–56, Oct. 2022, doi: 10.34306/itsdi.v4i1.569
  4. S. Suriyanto, A. Purkon, W. Aswawarman, T. S. Soeparyanto, and A. Kadir, "Performance Evaluation of Costs and Schedules for the Merah Putih Cooperative Construction Project Using Earned Value Management," ADI Bisnis Digit. Interdisiplin J., vol. 7, no. 1, pp. 105–116, Jun. 2026, doi: 10.34306/abdi.v7i1.1483
  5. N. P. L. Santoso, R. A. Sunarjo, and I. S. Fadli, "Analyzing the Factors Influencing the Success of Business Incubation Programs: A SmartPLS Approach," ADI J. Recent Innov., vol. 5, no. 1, pp. 60–71, Sep. 2023, doi: 10.34306/ajri.v5i1.985
  6. L. Larisang, S. Sanusi, M. A. Bora, and A. Hamid, "Practicality and Effectiveness of New Technopreneurship Incubator Model in The Digitalization Era," Aptisi Trans. Technopreneurship, vol. 7, no. 2, pp. 318–333, Jul. 2025, doi: 10.34306/att.v7i2.482
  7. A. Nuche, O. Sy, and J. Carlos Rodriguez, "Optimizing Efficiency Through Sustainable Strategies: The Role of Management and Monitoring in Achieving Goals," Aptisi Trans. Manage., vol. 8, no. 2, pp. 167–174, May 2024, doi: 10.33050/atm.v8i2.2257
  8. Sudaryono, U. Rahardja, and N. Lutfiani, "The Strategy of Improving Project Management Using Indicator Measurement Factor Analysis (IMF) Method," J. Phys.: Conf. Ser., vol. 1477, no. 3, Art. no. 032023, Mar. 2020, doi: 10.1088/1742-6596/1477/3/032023
  9. N. Lachlan, R. F. Terizla, and A. Lansonia, "Rethinking IMF: An Evolutionary Concept Analysis and Redefinition of Indicator Measurement Factor Analysis for Creative-Campus Projects," CreatiCampus Working Paper Series, vol. 1, no. 1, Esquine CreatiResearch Institute, Oct. 2026. [Online]. Available: https://creaticampus.esquine.com/papers/rethinking-imf-indicator-measurement-factor-analysis-creative-campus/
  10. E. A. Beldiq, G. Jacqueline, and A. Husain, "IMF as the Operating System of CreatiCampus Projects: A Process Framework from POAC to the Viewboard and the Creative-Campus Project Lifecycle," CreatiCampus Working Paper Series, vol. 1, no. 1, Esquine CreatiResearch Institute, Oct. 2026. [Online]. Available: https://creaticampus.esquine.com/papers/imf-operating-system-creative-campus-projects-poac-viewboard-lifecycle/
  11. N. Ani, A. Erica, and B. Callula, "Measuring What IMF Manages: A Design-Science Protocol for an Indicator Set and Maturity Model for CreatiCampus Projects," CreatiCampus Working Paper Series, vol. 1, no. 1, Esquine CreatiResearch Institute, Oct. 2026. [Online]. Available: https://creaticampus.esquine.com/papers/measuring-what-imf-manages-indicator-set-maturity-model-creative-campus/
  12. N. Lutfiani, U. Rahardja, and K. T. Khasanah, "The Development Viewboard As an Information Media at Official Site Asosiation," Aptisi Trans. Manage., vol. 6, no. 1, pp. 10–18, Jan. 2022, doi: 10.33050/atm.v6i1.1529
  13. D. Abbas, K. Siahaan, and M. Yusup, "Design Thinking as a Business Model for Empowering Creative Entrepreneurs in the Digital Era," Startupreneur Bus. Digit., vol. 4, no. 2, pp. 124–133, Oct. 2025, doi: 10.33050/sabda.v4i2.805
  14. F. Zidan, D. Nugroho, R. Asri, and S. Agustina, "Peran Pendidikan dalam Mempersiapkan Generasi Creativepreneur di Masa Depan," ADI Bisnis Digit. Interdisiplin J., vol. 5, no. 1, pp. 41–46, Jun. 2024, doi: 10.34306/abdi.v5i1.1099
  15. F. Rahardja, L. K. Choi, R. C. Wijaya, and R. A. Sunarjo, "Gamification in Digital Startups: Enhancing User Engagement and Business Growth," Startupreneur Bus. Digit., vol. 4, no. 1, pp. 1–11, Apr. 2025, doi: 10.33050/sabda.v4i1.639
  16. A. S. Bist, B. Rawat, U. Rahardja, Q. Aini, and A. G. Prawiyogi, "An Exhaustive Analysis of Stress on Faculty Members Engaged in Higher Education," IAIC Trans. Sustain. Digit. Innov., vol. 3, no. 2, pp. 126–135, Apr. 2022, doi: 10.34306/itsdi.v3i2.533
  17. T. K. Koo and M. Y. Li, "A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research," J. Chiropr. Med., vol. 15, no. 2, pp. 155–163, Jun. 2016, doi: 10.1016/j.jcm.2016.02.012
  18. M. H. R. Chakim, U. Rahardja, E. D. Astuti, E. Erika, and C. T. Hua, "The Social Empowerment Role of the Penta Helix Entrepreneurship Ecosystem in Driving the National Economy," ADI Pengabd. Masy. J., vol. 6, no. 1, pp. 1–13, Sep. 2025, doi: 10.34306/adimas.v6i1.1283
  19. R. Ginting Munthe, Q. Aini, N. Lutfiani, I. Van Persie, and A. Ramadhan, "Transforming Scientific Publication Management in the Era of Disruption: SmartPLS Approach in Innovation and Efficiency Analysis," Aptisi Trans. Manage., vol. 8, no. 2, pp. 123–130, May 2024, doi: 10.33050/atm.v8i2.2261
  20. P. E. Shrout and J. L. Fleiss, "Intraclass correlations: Uses in assessing rater reliability," Psychol. Bull., vol. 86, no. 2, pp. 420–428, 1979, doi: 10.1037/0033-2909.86.2.420
  21. K. O. McGraw and S. P. Wong, "Forming inferences about some intraclass correlation coefficients," Psychol. Methods, vol. 1, no. 1, pp. 30–46, Mar. 1996, doi: 10.1037/1082-989X.1.1.30
  22. A. F. Hayes and K. Krippendorff, "Answering the Call for a Standard Reliability Measure for Coding Data," Commun. Methods Meas., vol. 1, no. 1, pp. 77–89, Apr. 2007, doi: 10.1080/19312450709336664
  23. J. M. Bland and D. G. Altman, "Statistical Methods for Assessing Agreement Between Two Methods of Clinical Measurement," Lancet, vol. 327, no. 8476, pp. 307–310, Feb. 1986, doi: 10.1016/S0140-6736(86)90837-8
  24. S. Wibowo, I. A. Widjaya, J. Zanubiya, R. Evans, and U. Rahardja, "Orange Technology for Humanistic Innovation in Higher Education," J. MENTARI, vol. 4, no. 2, pp. 105–115, Mar. 2026, doi: 10.33050/mentari.v4i2.899
  25. U. Rahardja, N. Ani, and N. Leonie, "‘Inclusively Creative’ Campuses: An Orange Technology Framework for Humanistic Creative-Economy Higher Education in Indonesia," CreatiCampus Working Paper Series, vol. 1, no. 1, Esquine CreatiResearch Institute, Oct. 2026. [Online]. Available: https://creaticampus.esquine.com/papers/inclusively-creative-campus-orange-technology-wcce-2026/
  26. M. Hardini, M. Mardiana, D. Andayani, E. D. Astuti, and I. Maria, "Evaluation of Digital Branding Strategy in OJS-Based Journals to Increase Dissemination of Scientific Publications," ADI J. Recent Innov., vol. 8, no. 1, pp. 1–12, Sep. 2026, doi: 10.34306/ajri.v8i1.1495
  27. E. P. Lestari, A. Valerry, and L. Sulistyowati, "Measuring the Effect of Interdisciplinary Learning Factory Projects on Student Learning Outcomes," Int. Trans. Educ. Technol., vol. 4, no. 2, pp. 131–148, May 2026, doi: 10.33050/itee.v4i2.1109
  28. I. K. Mertayasa, U. Rahardja, M. Hardini, and E. P. Harahap, "Interior Design Alphabet Incubator 3.0 Based on Planner 5D," Technomedia J., vol. 8, no. 1, pp. 82–108, May 2023, doi: 10.33050/tmj.v8i1.1982