Metrics for Evaluation of the Chief Information Officer
The IT Steering Committee will evaluate the state Chief Information Officer (CIO) based on the metrics below, organized under seven priority areas. The evaluation summary will be included in the CIO’s annual report, due 20 days prior to the start of the 2018 Legislature.
IT Governance Metrics (2017)
| Metric Description | Measurement | Deadline | Status | Grade |
|---|---|---|---|---|
| Achieved full participation by executive branch departments in IT governance processes, pursuant to Administrative Directive (AD) No. 15-02, “Program Governance Requirements for Act 119 and Enterprise Information Technology Projects” | Percentage of departments participating | 07/1/17 | 100% of executive branch departments are participating (excluding the Department of Education and University of Hawaii). ETS continues to refine the IT “request to spend process” to enhance the depth and effectiveness of reviews of departmental hardware, software, telecom and service requests of $100,000 or more. Departments submit their information to ETS via a detailed form. Initial results have yielded elimination of unnecessary items/services, improvement of the specificity of project timelines and plans, and, in some cases, renegotiation of proposed contract terms, conditions and prices in the favor of the state. ETS is currently working on streamlining the IT request to spend process. | A |
| Achieved full participation by executive branch departments in annual budget request process, developed in partnership with the Department of Budget and Finance (B&F) in accordance with HRS section 27-43, requiring departments to develop and maintain their respective multi-year IT strategic and tactical plans and roadmaps as part of the state’s overall IT strategic plans | Percentage of departmental roadmaps submitted | 12/1/17 | ETS has collected preliminary roadmaps from all applicable departments and is continuing their refinement. ETS initiated a department by department effort to assist them in the development and refinement of their roadmaps. ETS has also worked with the Department of Budget and Finance (B&F) to incorporate the roadmaps into the IT governance and budget planning purposes. If a department’s spending request is not in its budget request, ETS will not approve the spending request without sound justification. In addition, B&F’s budget preparation instructions provided to departments for the 2018 Legislature require that all budget requests for IT and telecommunications services require the approval of the CIO. ETS completed review of IT-related budget requests for the FY19 Supplemental Budget and provided feedback to B&F. | A |
| Reduced costs of IT projects reviewed under IT governance processes (AD No. 15-02) | Percentage of reduction relative to overall IT costs; 2017 ETS goal: 5 percent | 12/1/17 | On track. Calculation underway. Review has been expanded to include IT and telecommunications purchases $50,000 and above. | A |
| Deployed citizen-facing website for state IT policies and standards; continuous deployment and maintenance of statewide policies and standards | Yes or no and overall quality of resource | 12/1/17 | On track. Three security policies and one set of standards will be presented at the November 2017 IT Steering Committee meeting for review. | A |
| Average Grade | A |
IT Workforce Development Metrics (2017)
| Metric Description | Measurement | Deadline | Status | Grade |
|---|---|---|---|---|
| Reduced vacancies within ETS | Percentage; 2017 goal: vacancies less than 10 percent | 12/1/17 | ETS currently has 22 vacancies (two of which will be filled after ETS’ proposed reorganizational structure is approved, as these positions will be redescribed) and five anticipated retirements at the end of the year. ETS also anticipates filling between two and four more positions before the end of the year. | C |
| Facilitate programs designed to establish, expand and/or promote career path opportunities within the State for IT workers: | ||||
| 1) Completed LinkedIn Pilot Project and demonstrated progress toward vacancy goal | Number of positions filled via LinkedIn; pilot project goal: 10 | 04/1/17 | ETS was able to hire or assign start dates for 13 new employees (42% of its vacancies) during the LinkedIn pilot project (October 2016 - April 2017), compared to 6 (17%) during the same timeframe a year prior. Furthermore, four additional candidates sourced through LinkedIn during the pilot have been identified for hire but are pending final approval. ETS has since executed a year-long agreement with LinkedIn to support the office’s ongoing recruitment efforts and highly recommends that other departments and agencies pursue similar agreements if their budgets allow. | A |
| 2) Achieved employee participation in Civil Service IT Broadbanding Pilot Project | Number of ETS employees participating in program; goal: 10 percent of applicable civil service IT employees | 12/1/17 | 9% of eligible ETS employees have participated in IT Broadbanding, applying for a Professional Growth adjustment or Internal Alignment. This has remained a steady number of participation since it was opened to employees. 5% of those who applied were awarded an IT Broadband Adjustment, a 2% increase from prior submittals. ETS has also used the salary matrix for at least two IT hires in 2017. | B |
| 3) Continued and expanded Hawaii Annual Code Challenge program | Held second annual event (yes or no) and quality of event compared to previous; goal: greater than 250 participants | 12/1/17 | ETS held the second HACC in partnership with the Hawaii Technology Development Corporation, nonprofit Hawaii Open Data, and DevLeague to engage the local tech community in government modernization and expose individuals within Hawaii’s IT workforce to State career opportunities. Approximately 300 attended the kickoff for the month-long competition on Aug. 26, 2017, answering the call to apply their collective creativity to improving state government. This is a significant increase over 220 in 2016. In addition, a number of proofs of concept from last year’s competition are in development and scheduled for launch this year. For example, the Hawaii Agriculture & Food Products Database, unveiled at the Hawaii Agriculture Conference in August 2017, was developed collaboratively by the Hawaii Department of Agriculture and ETS, incorporating input from several 2016 HACC solutions as well as past Transformation Internship Program cohorts. | A |
| Average Grade | B | |||
Cybersecurity Metrics (2017)
| Metric Description | Measurement | Deadline | Status | Grade |
|---|---|---|---|---|
| Filled all available ETS cybersecurity positions | Yes or no and/or percentage filled | 12/1/17 | Filled 100% of FY17 positions. Reviewing candidates for positions created or redescribed in FY18. | A |
| Implemented cybersecurity response plan | Yes or no | 12/1/17 | In progress | I |
| Established metrics for cybersecurity response and effectiveness | Yes or no and quality of metrics | 12/1/17 | In progress | I |
| Average Grade | I |
Enterprise Projects & Programs Metrics (2017)
| Metric Description | Measurement | Deadline | Status | Grade |
|---|---|---|---|---|
| Issued RFP for State Web Portal Program | Yes or no | 12/1/17 | A Request for Information (RFI) for an Internet Portal Manager and Service Provider was issued on July 12, 2017. Issuance of a Request for Proposals (RFP) is pending. | I |
| Demonstrate successful implementation of the following enterprise initiatives: | ||||
| 1) Tax System Modernization Project, Phases 2 & 3 | IV&V assessment as of date | 12/1/17 | In August 2017, the Tax System Modernization program completed the third of five “Rollouts,” which enhances online services for Corporate Income, Franchise, Public Service Company, and Withholding taxpayers. New features include improved web interface and added video tutorials. While the August 2017 launch requires no immediate action by tax-filers, it lays the necessary groundwork for phasing out the aforementioned tax group services from the legacy website by January 2018. ETS is also identifying any opportunities for further improvement before the project proceeds to its next major phase, Rollout 4, which will begin to address individual income filing and other areas. | A |
| 2) Enterprise Payroll and Time & Attendance Modernization, Payroll Phase | On track, on budget | 12/1/17 | The HawaiiPay project is being implemented in two phases: Payroll (2016-2018), followed by Time & Attendance. As part of the Payroll Phase, the first two departments are scheduled to “go live” on the new system in second quarter 2018, with remaining departments to begin transitioning in third quarter 2018. Project team efforts are currently focused on system configuration and departmental IT readiness to ensure that the required hardware, software and network bandwidth are in place when the system goes live. | A |
| 3) Kauhale On-Line Eligibility Assistance (KOLEA), Phases 1 & 2 | On track, on budget | 12/1/17 | In progress | F |
| Demonstrate sustainability and integration (with other ERP modules) of: | ||||
| 1) Human Resources Management System | Quality of sustainability and integration | 12/1/17 | In May 2017, ETS migrated the existing executive branch Human Resource Management System (HRMS), maintained and operated by DHRD, to primary and secondary Cloud-enabled data centers hosted by project partner CherryRoad Technologies. | A |
| Average Grade | B | |||
Services-Oriented Infrastructure Metrics (2017)
| Metric Description | Measurement | Deadline | Status | Grade |
|---|---|---|---|---|
| Increased capability to provide tech support to departments as enterprise service | Job tickets received and closed | 12/1/17 | 7229 of 7280 job tickets were closed in fiscal year 2017. | B |
| Demonstrate progress and success of: | ||||
| 1) Office 365 Project | Quality of departmental participation and sustainability | 07/1/17 | ETS has completed migration of the executive branch to Office 365, excluding the Department of Taxation (pending its migration, which it elected to defer), Department of Education, University of Hawaii and Hawaii Health Systems Corporation. ETS has also formally identified Office 365 as the state’s platform for email, online storage and other Office applications and services, and has negotiated a new 3-year enterprise license agreement (ELA), effective June 1, 2017, that provides for the continued use of these applications and services by approximately 12,000 users, while further leveraging economies of scale in favor of the state. | B |
| 2) eSign Service | Number of transactions and quality of departmental participation; year-end goal: 200,000 transactions | 12/1/17 | As of Nov. 16, 2017, there has been more than 263,410 total eSign transactions. The average number of eSign transactions per month is 9,931, with the record month reaching 19,310. The median time to complete a transaction is 238.2 minutes. | D |
| 3) Government Private Cloud / Cloud Services | Migration completed (yes or no) | 12/1/17 | Pending | F |
| 4) Network Operations & Maintenance | Reliability and sustainability; goal for core Next Generation Network: 99.99% availability, excluding planned maintenance | 12/1/17 | Calculation underway. Added network redundancy to Hawai‘i Emergency Management Agency (HI-EMA) statewide network to support both HI-EMA and O‘ahu INET users, performed network topology changes to increase the reliability of the State’s network, established a dedicated connection (express route) to the Microsoft Azure Cloud, and provided a high-speed fiber optic connection to the ‘Ewa District Court. Also, established new statewide price list through the State Procurement Office for Centrex Telephone and Cabling Services, and consolidated and significantly increased network bandwidth (200-fold) for tenants of the Kona State Office Building and the Molokai State Office Building. | B |
| 5) Telecommunications | Successful 99.99% availability, excluding planned maintenance | 12/1/17 | Calculation underway. | B |
| Average Grade | B | |||
Open Data Metrics (2017)
| Metric Description | Measurement | Deadline | Status | Grade |
|---|---|---|---|---|
| Launched ETS Strategic Roadmap Dashboard | Yes or no | 07/31/17 | Yes; see ETS IT Strategic Roadmap | A |
| Established web accessibility standard and launched statewide training resources | Yes or no | 12/1/17 | Pending | F |
| Established policy addressing copyright of State web content | Yes or no | 12/1/17 | ETS published State of Hawaii Government Website Terms of Use for official state websites that are not part of the eHawaii Internet Portal Program and/or for which the contractor-provided terms are not appropriate. | A |
| Evaluated effectiveness of State Web Portal program and model, and issue request for proposals in line with the recommendations of the Access Hawaii Committee | Yes or no | 12/1/17 | In progress. A Request for Information (RFI) for an Internet Portal Manager and Service Provider was issued on July 12, 2017. Issuance of a Request for Proposals (RFP) is pending. | I |
| Established policy for open data | Yes or no | 12/1/18 | Planning stage | I |
| Average Grade | I |
Cost-Transparency Metrics (2017)
| Metric Description | Measurement | Deadline | Status | Grade |
|---|---|---|---|---|
| Collected departmental IT roadmaps, under new authority provided under HRS 27-43 | Quality of departmental participation | 07/1/17 | ETS completing the process of initial reviews with departments. | I |
| Created ETS IT strategic roadmap, incorporating and aligning information from departmental IT roadmaps | Yes or no | 07/1/17 | Yes; however, incorporating/aligning with department roadmaps is ongoing as departments develop their individual roadmaps, described above. | A |
| Published ETS IT strategic roadmap setting clear goals and benchmarks for the CIO’s priority areas and priority projects/programs by the deadline to submit Executive Budget Request to the Legislature | Yes or no | 07/1/17 | Yes; see ETS IT Strategic Roadmap | A |
| Created a publically accessible online dashboard to track progress, as a companion to the roadmap | Yes or no | 12/1/17 | ETS is in the process of creating the dashboard, summarizing all departmental IT roadmaps. | F |
| Average Grade | B |