In early 2022, a multidisciplinary HSRC product team piloted a custom-built online platform to improve the collection and curation of South Africa’s research and development statistics. Gerard Ralphs, Jacqueline Borel-Saladin and Nazeem Mustapha share lessons from this public-sector innovation project.

For the past 21 years, the HSRC’s Centre for Science, Technology and Innovation Indicators (CeSTII) has performed the National Survey of Research and Experimental Development Inputs (R&D Survey) for the Department of Science and Innovation. The survey is vital for measuring R&D progress in South Africa across the main sectors of government, business, non-profit, science councils and higher education. Government leaders, policymakers and researchers as well as intergovernmental organisations, such as the Organisation for Economic Cooperation and Development (OECD) and the United Nations Educational, Scientific and Cultural Organization (UNESCO), routinely use the data for decision-making and monitoring purposes.

As official statistics, the R&D Survey must maintain high standards of quality, especially a high response rate. Recently, the response rate dropped, most significantly in the vital business sector component of the survey. The reasons are varied. COVID-19 and the related restrictions made re-establishing and maintaining contact with respondents challenging. Many of these respondents were working from home, which meant that their contact details had changed, and they often had no documentation with them to complete the data requested in the survey questionnaire. But even before the onset of COVID-19, the rate of refusals was increasing.

Under the appropriate conditions, it has been found that web-based questionnaire collections can improve response rates in enterprise surveys. Such conditions appear to pertain to the R&D Survey, with the expectation being that businesses with the potential of doing R&D are also very likely to be enabled to respond through an online mechanism. In other sectors, such as government, where some institutions adhere to compliance processes that include hard copies of documentation and correspondence, perhaps not so much. But even here, the move towards digitalised data collection appears to be better supported and generally more accepted. Finally, perhaps more specific to the administration of the R&D Survey, fieldworkers spend an inordinate amount of time on paper processing and data capturing. These and other operational bottlenecks, some introduced as a result of COVID-19 but others of a legacy nature, constrain the ability of researchers to derive value from the data collected. All this takes place in an environment where policymakers have called repeatedly for more analysis of the survey results.

The rationale of the digitalisation business case for the R&D Survey, therefore, is that enabling online survey participation can help the HSRC to achieve multiple objectives. From the perspective of respondent users, the most important goals are enhancing the survey data submission experience, improving the reputation of the survey through two-way data flows, and strengthening public participation in R&D. For the research team users, greater online participation by respondents can significantly reduce the internal burden of manually capturing data and, in turn, limit the lead time between data collection and the timely release of results for public use by decision makers.

Technologically, the product team conducted research on suitable ‘tech stacks’ before an architectural solution was proposed. Through this, both legacy systems and open-source applications were integrated into the final design.

Photo: Mikael Blomkvist, Pexels

Learning moments

Trying to achieve all these goals in a single product innovation was an ambitious task, requiring almost a full year of design thinking and prototype build sprints to reach the first release. During the pilot in 2022, we achieved some operational successes, reinforcing our business case. But we also anticipated, and experienced, some teething problems. In this context, the product team employed the principles of design thinking and agile development and, as a result, problems could be addressed quickly, as the team instituted not just temporary fixes but lasting solutions as the system was being used. Thanks to this approach, several users reported that the online system was easy to navigate with excellent support (see Box 1).

Overall, the conversion of respondents to the online platform was only partially achieved during the pilot: about one-fifth of all respondents in the survey frame. However, in certain sectors, online participation was higher, with just under one quarter of respondents from higher education institutions submitting responses online, as did nearly one third of non-profit organisations. While there was a steady increase in responses over the survey period, there were also noticeable spikes in response to communication campaigns aimed at reminding and encouraging respondents to complete the survey. Interestingly, towards the end of the survey period, several users opted for the more familiar PDF form version of the questionnaire over the online system.

The team is working on additional value-added data services to attract more online respondents in 2021/2022. These include guidance on where respondents can engage with their firm’s historic survey data through graphical widgets or downloadable PDFs. Equally, the platform’s researcher interface is currently being developed so that researchers can monitor the progress of the survey better in real time and be more proactive in engaging with respondents.

As with many other digital contexts, introducing a new way of doing things through the adoption of the new platform may require several iterations. While the R&D Survey online platform did not achieve maximum take-up, important lessons have been learnt in the first year of its in-field deployment. Already, using the new software in the field has provided important feedback for the refinement of the R&D Survey respondent experience, the researcher interface and the overall goal of end-to-end digitalisation.

Public sector, public value

Despite it being almost cliché, the Silicon Valley adage ‘fail first, fail fast’ succeeds in conveying the vital role of learning in innovation. To fail is to learn; to fail quickly is to speed up learning, a way to get ahead of the curve. Elon Musk’s famous injunction to SpaceX workers similarly turns failure into a pathway to be actively pursued: ‘Failure is an option here. If things are not failing, you are not innovating enough.’ In public-sector innovation practice in South Africa, at least in our experience, getting ahead of the curve may not be a business competition problem as much as it is a challenge in overcoming public sector constraints and ensuring that our constituents – principally taxpayers – recognise public value early in the innovation process. In the context of the new R&D Survey online platform, key lessons learnt through piloting have paved the way for further improvements in preparation for the next survey round.

Learn more about the R&D survey

Authors: Dr Jacqueline Borel-Saladin, senior research specialist; Gerard Ralphs, programme manager; and Dr Nazeem Mustapha, chief research specialist, in the HSRC’s Centre for Science, Technology and Innovation Indicators

jmborel-saladin@hsrc.ac.za

gralphs@hsrc.ac.za

nmustapha@hsrc.ac.za