Metrics for Evaluation of the Chief Information Officer

The IT Steering Committee will evaluate the state Chief Information Officer (CIO) based on the metrics below, organized under seven priority areas. The evaluation summary will be included in the CIO’s annual report, due 20 days prior to the start of the 2018 Legislature.


IT Governance Metrics (2017)

Metric DescriptionMeasurementDeadlineStatusGrade
Average Grade
Achieved full participation by executive branch departments in IT governance processes, pursuant to Administrative Directive (AD) No. 15-02, “Program Governance Requirements for Act 119 and Enterprise Information Technology Projects”Percentage of departments participating07/1/17100% of executive branch departments are participating (excluding the Department of Education and University of Hawaii). ETS continues to refine the IT “request to spend process” to enhance the depth and effectiveness of reviews of departmental hardware, software, telecom and service requests of $100,000 or more. Departments submit their information to ETS via a detailed form. Initial results have yielded elimination of unnecessary items/services, improvement of the specificity of project timelines and plans, and, in some cases, renegotiation of proposed contract terms, conditions and prices in the favor of the state. ETS is currently working on streamlining the IT request to spend process.
Achieved full participation by executive branch departments in annual budget request process, developed in partnership with the Department of Budget and Finance (B&F) in accordance with HRS section 27-43, requiring departments to develop and maintain their respective multi-year IT strategic and tactical plans and roadmaps as part of the state’s overall IT strategic plansPercentage of departmental roadmaps submitted12/1/17ETS has collected preliminary roadmaps from all applicable departments and is continuing their refinement. ETS initiated a department by department effort to assist them in the development and refinement of their roadmaps. ETS has also worked with the Department of Budget and Finance (B&F) to incorporate the roadmaps into the IT governance and budget planning purposes. If a department’s spending request is not in its budget request, ETS will not approve the spending request without sound justification. In addition, B&F’s budget preparation instructions provided to departments for the 2018 Legislature require that all budget requests for IT and telecommunications services require the approval of the CIO. ETS completed review of IT-related budget requests for the FY19 Supplemental Budget and provided feedback to B&F.
Reduced costs of IT projects reviewed under IT governance processes (AD No. 15-02)Percentage of reduction relative to overall IT costs; 2017 ETS goal: 5 percent12/1/17On track. Calculation underway. Review has been expanded to include IT and telecommunications purchases $50,000 and above.
Deployed citizen-facing website for state IT policies and standards; continuous deployment and maintenance of statewide policies and standardsYes or no and overall quality of resource12/1/17On track. Three security policies and one set of standards will be presented at the November 2017 IT Steering Committee meeting for review.

IT Workforce Development Metrics (2017)

Metric DescriptionMeasurementDeadlineStatusGrade
Average Grade
Reduced vacancies within ETSPercentage; 2017 goal: vacancies less than 10 percent12/1/17ETS currently has 22 vacancies (two of which will be filled after ETS’ proposed reorganizational structure is approved, as these positions will be redescribed) and five anticipated retirements at the end of the year. ETS also anticipates filling between two and four more positions before the end of the year.
Facilitate programs designed to establish, expand and/or promote career path opportunities within the State for IT workers:
1) Completed LinkedIn Pilot Project and demonstrated progress toward vacancy goalNumber of positions filled via LinkedIn; pilot project goal: 1004/1/17ETS was able to hire or assign start dates for 13 new employees (42% of its vacancies) during the LinkedIn pilot project (October 2016 - April 2017), compared to 6 (17%) during the same timeframe a year prior. Furthermore, four additional candidates sourced through LinkedIn during the pilot have been identified for hire but are pending final approval. ETS has since executed a year-long agreement with LinkedIn to support the office’s ongoing recruitment efforts and highly recommends that other departments and agencies pursue similar agreements if their budgets allow.
2) Achieved employee participation in Civil Service IT Broadbanding Pilot Project Number of ETS employees participating in program; goal: 10 percent of applicable civil service IT employees12/1/179% of eligible ETS employees have participated in IT Broadbanding, applying for a Professional Growth adjustment or Internal Alignment. This has remained a steady number of participation since it was opened to employees. 5% of those who applied were awarded an IT Broadband Adjustment, a 2% increase from prior submittals. ETS has also used the salary matrix for at least two IT hires in 2017.
3) Continued and expanded Hawaii Annual Code Challenge programHeld second annual event (yes or no) and quality of event compared to previous; goal: greater than 250 participants12/1/17ETS held the second HACC in partnership with the Hawaii Technology Development Corporation, nonprofit Hawaii Open Data, and DevLeague to engage the local tech community in government modernization and expose individuals within Hawaii’s IT workforce to State career opportunities. Approximately 300 attended the kickoff for the month-long competition on Aug. 26, 2017, answering the call to apply their collective creativity to improving state government. This is a significant increase over 220 in 2016. In addition, a number of proofs of concept from last year’s competition are in development and scheduled for launch this year. For example, the Hawaii Agriculture & Food Products Database, unveiled at the Hawaii Agriculture Conference in August 2017, was developed collaboratively by the Hawaii Department of Agriculture and ETS, incorporating input from several 2016 HACC solutions as well as past Transformation Internship Program cohorts.

Cybersecurity Metrics (2017)

Metric DescriptionMeasurementDeadlineStatusGrade
Average Grade
Filled all available ETS cybersecurity positionsYes or no and/or percentage filled12/1/17Filled 100% of FY17 positions. Reviewing candidates for positions created or redescribed in FY18.
Implemented cybersecurity response planYes or no12/1/17In progress
Established metrics for cybersecurity response and effectiveness Yes or no and quality of metrics12/1/17In progress

Enterprise Projects & Programs Metrics (2017)

Metric DescriptionMeasurementDeadlineStatusGrade
Average Grade
Issued RFP for State Web Portal ProgramYes or no12/1/17A Request for Information (RFI) for an Internet Portal Manager and Service Provider was issued on July 12, 2017. Issuance of a Request for Proposals (RFP) is pending.
Demonstrate successful implementation of the following enterprise initiatives:
1) Tax System Modernization Project, Phases 2 & 3IV&V assessment as of date12/1/17In August 2017, the Tax System Modernization program completed the third of five “Rollouts,” which enhances online services for Corporate Income, Franchise, Public Service Company, and Withholding taxpayers. New features include improved web interface and added video tutorials. While the August 2017 launch requires no immediate action by tax-filers, it lays the necessary groundwork for phasing out the aforementioned tax group services from the legacy website by January 2018. ETS is also identifying any opportunities for further improvement before the project proceeds to its next major phase, Rollout 4, which will begin to address individual income filing and other areas.
2) Enterprise Payroll and Time & Attendance Modernization, Payroll Phase On track, on budget12/1/17The HawaiiPay project is being implemented in two phases: Payroll (2016-2018), followed by Time & Attendance. As part of the Payroll Phase, the first two departments are scheduled to “go live” on the new system in second quarter 2018, with remaining departments to begin transitioning in third quarter 2018. Project team efforts are currently focused on system configuration and departmental IT readiness to ensure that the required hardware, software and network bandwidth are in place when the system goes live.
3) Kauhale On-Line Eligibility Assistance (KOLEA), Phases 1 & 2On track, on budget12/1/17In progress
Demonstrate sustainability and integration (with other ERP modules) of:
1) Human Resources Management SystemQuality of sustainability and integration12/1/17In May 2017, ETS migrated the existing executive branch Human Resource Management System (HRMS), maintained and operated by DHRD, to primary and secondary Cloud-enabled data centers hosted by project partner CherryRoad Technologies.

Services-Oriented Infrastructure Metrics (2017)

Metric DescriptionMeasurementDeadlineStatusGrade
Average Grade
Increased capability to provide tech support to departments as enterprise serviceJob tickets received and closed12/1/177229 of 7280 job tickets were closed in fiscal year 2017.
Demonstrate progress and success of:
1) Office 365 ProjectQuality of departmental participation and sustainability07/1/17ETS has completed migration of the executive branch to Office 365, excluding the Department of Taxation (pending its migration, which it elected to defer), Department of Education, University of Hawaii and Hawaii Health Systems Corporation. ETS has also formally identified Office 365 as the state’s platform for email, online storage and other Office applications and services, and has negotiated a new 3-year enterprise license agreement (ELA), effective June 1, 2017, that provides for the continued use of these applications and services by approximately 12,000 users, while further leveraging economies of scale in favor of the state.
2) eSign ServiceNumber of transactions and quality of departmental participation; year-end goal: 200,000 transactions12/1/17As of Nov. 16, 2017, there has been more than 263,410 total eSign transactions. The average number of eSign transactions per month is 9,931, with the record month reaching 19,310. The median time to complete a transaction is 238.2 minutes.
3) Government Private Cloud / Cloud ServicesMigration completed (yes or no)12/1/17Pending
4) Network Operations & MaintenanceReliability and sustainability; goal for core Next Generation Network: 99.99% availability, excluding planned maintenance12/1/17Calculation underway. Added network redundancy to Hawai‘i Emergency Management Agency (HI-EMA) statewide network to support both HI-EMA and O‘ahu INET users, performed network topology changes to increase the reliability of the State’s network, established a dedicated connection (express route) to the Microsoft Azure Cloud, and provided a high-speed fiber optic connection to the ‘Ewa District Court. Also, established new statewide price list through the State Procurement Office for Centrex Telephone and Cabling Services, and consolidated and significantly increased network bandwidth (200-fold) for tenants of the Kona State Office Building and the Molokai State Office Building.
5) TelecommunicationsSuccessful 99.99% availability, excluding planned maintenance12/1/17Calculation underway.

Open Data Metrics (2017)

Metric DescriptionMeasurementDeadlineStatusGrade
Average Grade
Launched ETS Strategic Roadmap DashboardYes or no07/31/17Yes; see ETS IT Strategic Roadmap
Established web accessibility standard and launched statewide training resourcesYes or no12/1/17Pending
Established policy addressing copyright of State web contentYes or no12/1/17ETS published State of Hawaii Government Website Terms of Use for official state websites that are not part of the eHawaii Internet Portal Program and/or for which the contractor-provided terms are not appropriate.
Evaluated effectiveness of State Web Portal program and model, and issue request for proposals in line with the recommendations of the Access Hawaii CommitteeYes or no12/1/17In progress. A Request for Information (RFI) for an Internet Portal Manager and Service Provider was issued on July 12, 2017. Issuance of a Request for Proposals (RFP) is pending.
Established policy for open dataYes or no12/1/18Planning
stage

Cost-Transparency Metrics (2017)

Metric DescriptionMeasurementDeadlineStatusGrade
Average Grade
Collected departmental IT roadmaps, under new authority provided under HRS 27-43Quality of departmental participation07/1/17ETS completing the process of initial reviews with departments.
Created ETS IT strategic roadmap, incorporating and aligning information from departmental IT roadmapsYes or no07/1/17Yes; however, incorporating/aligning with department roadmaps is ongoing as departments develop their individual roadmaps, described above.
Published ETS IT strategic roadmap setting clear goals and benchmarks for the CIO’s priority areas and priority projects/programs by the deadline to submit Executive Budget Request to the LegislatureYes or no07/1/17Yes; see ETS IT Strategic Roadmap
Created a publically accessible online dashboard to track progress, as a companion to the roadmapYes or no12/1/17ETS is in the process of creating the dashboard, summarizing all departmental IT roadmaps.