The deliverable 7.1, Approach and Methodology for Monitoring and Evaluation, was submitted by Future Analytics Consulting in June 2019 and revised in May 2020. Following, the executive summary of the deliverable:
“This report presents Deliverable 7.1 – “Approach and Methodology for Monitoring and Evaluation” of WP 7 of the +CityxChange project. The task was set to develop a standardised approach and methodology to Monitoring and Evaluation (M&E) that could be applied to measure the impact of all interventions implemented through the demonstration projects.
This report provides a detailed M&E standardised approach applicable to all eleven Demonstration Projects and the overall project impact, undertaken in close conjunction with the LHCs, FCs, and Solution Providers. This approach will incorporate the defined Key Performance Indicator (KPI) Framework (Section 1.3.6.2) (based on existing Smart Cities projects and programmes and extended with KPIs specific to the +CityxChange project). Moreover, the standardised approach includes transformation of the incoming measurements to KPI measures into a M&E scoring output. In addition, the results from a workshop held during the first consortium meeting will be described. Finally, the report will specify periodic updates of the KPI Framework to ensure most relevant KPIs are being captured and the frequency of collection is appropriate (connected to Task 7.1).
The data requirements for the M&E process were considered by looking into the quality and quantity, focussing on the provision of metadata and project monitoring data through the process of data collection for the different interventions specified by the project. A definition of the parties responsible for the data collection process was derived by looking into the project partners grouped into KPI owners and/or data owners where relevant. The distinction between the relevant KPI and data owners are based on the key responsibilities of each partner – as KPI owners lead the implementation and monitoring of the intervention and take responsibility for its impacts, and data owners provide technical support and expert insight into the data management process. The governance of data is also discussed in line with the data collection process as the parties responsible will need to adhere to the relevant GDPR requirements, Creative Commons data sharing, and H2020 protocols for fair data management and dissemination. The project aims to share as much data as possible with the public while ensuring that sensitive and personal data is protected in line with its Data Management Plan.
Further to the data collection process the relevant project boundaries are defined, using a macro to micro overview, in order to understand the spatial scale at which data is captured and reported. Examples provided give further insight in the way data will be aggregated for the purpose of reporting at a certain scale. To apply the M&E approach in the +CityxChange project, the consortium developed a framework of KPIs, of which some relates to the SCIS, while others were developed to measure specific KPIs that were developed for the project. The document discusses the definition of a KPI, how it is used in the project, and the learnings and initiatives considered in setting up the project’s KPIs. More emphasis is placed on the use of KPIs related to the SCIS and the specific KPIs developed for the project.
Sections 6 and 8 of the deliverables respectively refer to the process of standardisation of project monitoring data to be followed in project M&E, and the importance of maintaining good standards of M&E through reviewing the process of collection and relevance of data capturing procedures and KPIs. A review process is proposed which is to be followed if deemed that a KPI should be amended.
As an annex to the document the KPI information tables are included for the perusal of KPI & data owners. The tables provide an overview of the definition, description and detailed calculation parameters of each of the 33 KPIs. The tables were designed as a ‘go-to’ reference for all partners needing more info on the calculation procedures for their respective KPIs.”
