The deliverable 7.3, Data Collation, Management and Analysis Methodology Framework, was submitted by Future Analytics Consulting in October 2019. Following, the executive summary of the deliverable:
“To ensure consistent and accurate and reliable data analysis over, and beyond, the five year life cycle of the +CityxChange project, it is important to have an established methodology for monitoring and evaluation (M&E) of project data measured from the implementation of project interventions.
The data to be monitored is generated at the source of the planned intervention implemented in the project area, and can be monitored using sensors, online data collection systems, surveying or other monitoring mechanisms suitable to the intervention. The Key Performance Indicator (KPI) framework refined in Deliverable 7.1 – Approach and Methodology for Monitoring and Evaluation, is used to guide KPI and data owners on the parameters and variables to consider in the calculation of the indicator, when data is to be reported, and the spatial scale at which the data is to be reported. These parameters were used in the setup of the M&E Reporting Tool (MERT) to determine the calculations required for each KPI. The MERT forms part of the Information and Communication Technology (ICT) Ecosystem of the +CityxChange project, and provides a repository for monitoring data captured by KPI and data owners, from where the data is modelled, displayed and made available for further dissemination.
KPI and data owners are responsible to manage the monitoring data captured in their own repositories or active monitoring systems. Here, the data will be prepared for submittal to the monitoring tools. If required, and dependent on the indicator, the KPI/data owner will process the captured data to the format required for the calculation before it is submitted. The submission of data can then take place through a manual process (submitting data through the online KPI interface of the MERT or an automated process (sharing data between the partner’s data portal and the MERT through API connection). The availability of data for monitoring and display in the MERT is however dependent on the implementation of interventions, and will be informed by the KPI/data owners. To promote wider data dissemination, monitoring data for six KPIs will also be reported to the Smart Cities Information System (SCIS) Self-Reporting Tool (SRT).
In support of the monitoring data captured in the MERT, partners will have the option of uploading documents that provide further information on the data that is captured. The supporting documentation can include detailed calculations or processing applied to the data captured, more granular levels of data, links to other deliverables describing the interventions for which data is captured and other supporting information that does not include any personal information.
The data that is captured, modelled and displayed in the MERT, but also made available for download to enable external use. The data is made available to the public domain to comply with the European Commission (EC) regulations and +CityxChange Data Management Plan (DMP) of open access to data, and making data Findable, Accessible, Interoperable, and Re-usable (FAIR). The data that is made available for download from the MERT and is shared under Creative Commons Licensing conditions to ensure data dissemination with the necessary attribution. The MERT will only allow accredited users (KPI/data owners) access to modify (add and edit) data and download of all the data available. Public users will have access to monitoring data, being able to download the aggregated data for the KPI, and a ‘.pdf’ summary report, whilst not able to modify any of the captured data. The data displayed in the MERT’s individual KPI interface will include the KPI number, description, expected or targeted impact, unit of measurement, frequency of reporting, annual performance of the KPI presented as a percentage, visual representation of data (graphs), and other key information relevant to each KPI. From this interface, the accredited users will be able to modify data, and accredited and public users alike will be able to generate the summary report.
As it is still unclear when monitoring data will be available for many KPIs, the development of the MERT employs a flexible structure approach to allow for future integration and possible alterations. The MongoDB NoSQL database management structure on which the MERT is based, accommodates a variety (and large volumes) of data and allows easy scalability. The two methods of submitting data to the MERT further enhances its long term and ease of usability. To promote the use of the automated process of data sharing, the APIs were structured using the FIWARE guidelines for KPIs in smart city projects, making the later integration and use of this method by partners easier and more efficient.
As the project progresses, it is of critical importance to receive partner’s feedback on the use of the MERT, and to engage on ways to improve the efficiency and use of the tool to assist partners in achieving project goals. Although this deliverable sets the foundation on which M&E in the tool is built, the continuous improvement of the tool is dependent on a collaborative effort between all the partners involved.
