Managing records as the evidence base for development: access and integrity in the digital environment

Anne Thurston ANALYSIS Public Administration & Governance

Every public sector policy maker, auditor, court official and criminal investigator knows the importance of being able to find, use and trust official records as evidence of policies, actions, transactions, expenditure, precedents and rights. And yet, little if any attention is being given to the challenges of managing the records produced in the digital environment as reliable evidence of what governments have promised, what they have done, what they have spent, what services they have provided and how citizens’ rights have been protected.

Goals for sustainable development rest on the assumption that trustworthy digital information will be available, can be shared meaningfully and can be used effectively through strategies for digital governance. This assumption needs to be examined, and policy makers and development planners in Africa and elsewhere need to be able to determine whether digital records in their countries can be accessed and trusted as reliable evidence.

This paper addresses the following five basic questions: What are the risks for digital records? What is the impact when digital records are not managed? How are high-quality digital records created and preserved? What are the benefits of high-quality digital records? What are the challenges for the future?

What are the risks for digital records?

Managing paper records presents significant ongoing challenges, but the challenges of managing digital records bring greater risks. The seven high-level risks for digital records described below result in cumulative losses that have significant consequences for any government’s ability to achieve transparency and accountability and to measure compliance and progress toward development goals.

Risk One: Lack of awareness that ICT systems create records

Many government stakeholders and development planners are not aware that ICT systems create records but generally do not have the full functionality needed to capture and retain them as accurate evidence.

Risk Two: Lack of structures and capacity for managing digital records

In many cases, the legal and regulatory framework and the capacity needed to manage digital records and data are not in yet place. As a result, digital records can be altered, deleted, fragmented, corrupted or lost inadvertently.

Risk Three: Digital records can be difficult to access and share

Digital records are often stored on personal drives, un-networked computers, or unmanaged network drives, making them unavailable as a corporate resource. Different versions of the same records can be kept without adequate identification.

Risk Four: Essential metadata is not being captured systematically

Metadata is essential to provide evidence of how the records were created, managed and used and of their relationship to other records. Once a record is registered and its context is fixed, it is very difficult to alter it or delete it without detection. Anything that happens to the record subsequently should also be documented through metadata, thus providing an audit trail of changes in status that makes it possible to identify fraud or illegal actions.

Risk Five: Procedures and facilities for preserving digital records do not exist

Digital media deteriorate and system documentation is lost if digital records are not transferred to secure preservation facilities where they can be managed over time.

Risk Six: Digitized records lose their legal value and their usability if they are not managed

Initiatives for digitizing paper records often fail to incorporate requirements for legal admissibility, reliability and usability, for instance, requirements for metadata capture, image resolution and classification structure. When this happens, digitized records are difficult to retrieve and use, and they lack legal value.

Risk Seven: Poor-quality data has limited value

High-quality data can be used to improve policy decisions and services; support research, planning and monitoring; and empower citizens. However, when public sector data is not protected and preserved systematically, it can easily be lost and its value diminished. Without accurate and consistent metadata to provide contextual information, data may have little or no real value.

What is the impact when digital records are not managed?

When digital records are not managed, they do not remain available and they lack integrity. Availability is achieved when a government or an organization manages and protects digital records to ensure their timely, efficient and accurate retrieval. Integrity is achieved when records are managed through a consistent regulatory framework that enables citizens, government stakeholders, investors and development agencies to trust the evidence provided. Four primary governance goals, transparency, accountability, the rule of law, and social inclusion, which are essential for achieving global development goals, cannot be achieved if digital records cannot be accessed or are unreliable.

Transparency

Transparency involves the release of evidence about a government’s decisions, business processes and activities in such a way that it is easy for citizens, investors, and donors/ lenders to observe how the government has performed and used resources.

• Availability: When records are incomplete, corrupted, cannot be accessed or are lost completely, policies, expenditure and transactions cannot be transparent.

• Integrity: When records do not provide authentic and credible evidence as a basis for developing policy or for planning, resources are wasted and citizens lose faith in the information their governments provide.

Accountability

Accountability involves the ability of a government or an agency to provide reliable evidence that its policies, decisions, actions and transactions are subject to oversight by auditors, investigators and regulatory bodies. The aim is to ensure that public sector initiatives are meeting their stated objectives, responding to the needs of the community they are meant to serve and providing value for money in public services.

• Availability: When records are not available to demonstrate accountability, trace corruption, enable audits and monitoring exercises, support citizens’ rights and entitlements, meet RTI requests and provide the basis for reliable open data, accountability cannot be established.

• Integrity: When records are not reliable, audits and investigations are flawed, the right to information is undermined and citizens cannot rely on the data the government releases.

Rule of Law

The rule of law involves ensuring that government officials, individuals and private entities are accountable under the law; that laws are clear, publicized, stable and just; and that they are applied fairly and efficiently to guarantee fundamental rights, including the security of persons and property. The rule of law also involves ensuring that competent and independent individuals deliver justice in a timely manner.

Availability: When records are not available, misconduct or miscarriage of justice cannot be proven or addressed, case precedent is difficult to establish, and it is not possible to demonstrate that the government or organization has complied with laws, regulations and organizational policies.

Integrity: When records are not reliable, theft of state assets cannot be traced and citizens’ rights and entitlements can be manipulated for corrupt purposes.

Social Inclusion

Social inclusion is the process of improving the terms on which all people take part in society while removing stigmatized markers that exclude individuals or groups from processes and opportunities. It aims to empower poor and marginalized people to have a voice in decisions that affect their lives, to claim their rights, and to have equal access to markets, services and expanding global opportunities.

Availability: If records of decisions and activities cannot be accessed, it is not possible to monitor and demonstrate progress toward social inclusion or to enable disadvantaged groups to demonstrate exclusion or denial of rights.

Integrity: If records are not reliable, progress toward social inclusion goals cannot be measured in a meaningful way, and the legal rights of marginalized communities and individuals cannot be protected.

How are high-quality digital records created and preserved?

Building high-quality evidence involves developing and implementing a framework of interconnected laws, policies, standards, procedures, responsibilities and capacities for creating and preserving digital records. A simple set of high-level questions will help to determine whether the necessary framework is in place:

• Is there a law that establishes the requirements for managing records, regardless of media or format, from creation to destruction or preservation as archives? Is the law harmonized with other relevant laws and regulations, for instance laws governing security, privacy, digital governance, the Right to Information and Open Data?

• Is there a high-level policy that establishes requirements for creating, managing and preserving records and their metadata across the government or the organization? Has the policy been harmonized with policies for digital governance, Open Data and Right to Information?

• Have international or national standards been implemented to provide common and consistent guidance on the structure and management of records for the entire government? Are the standards understood and consistently applied?

• Are senior government stakeholders aware that digital administrative systems create digital records, and have their responsibilities for these records been clearly defined at all levels across the government or the organization?

• Is there a central agency or authority with responsibility for oversight of the quality of digital records and data?

• Do ICT and records authorities collaborate to define, implement, and audit sustainable good practice for managing digital records and data?

• Are procedures in place to:

1. ensure that appropriate metadata are captured for government records and data?

2. define how long the records and data need to be kept to support legal, regulatory, fiscal and operational needs?

3. capture and retain email in a record-keeping system?

4. protect security of records and prevent unauthorized access, alteration or deletion?

• Are records professionals trained, in universities or management institutes, to understand national policy and international standards for creating, managing and preserving digital records?

• Do digitization initiatives incorporate requirements for legal admissibility, reliability and usability?

• Are adequate finances available as part of routine operational budgets to consistently provide qualified staff and facilities needed to manage and monitor the availability and integrity of digital records across the government?

What are the benefits of high-quality digital records?

• Digital records can be opened to the public systematically and privacy rights can be protected. The public can trust the records their governments create and use.

• Right to Information requests can be met rapidly and reliably. Restrictions can be justified legally, and documents can be tagged when there is a restriction on release.

• Public sector agencies can use the records confidently to plan and monitor programmes, activities and expenditure. They can trace, relate and compare policies, decisions, actions and expenditure accurately over long periods of time and identify precedents as a basis for an informed and socially just society.

• Accountability and transparency can be safeguarded and demonstrated. The metadata audit trail of changes to, or unauthorized use of, digital records provides a means of tracking fraud.

• Digital records, along with their essential metadata, can be securely and systematically extracted from diverse digital systems and transferred to long-term digital custody, where they can continue to meet legal, administrative, fiscal, or other evidentiary needs through time. They can be migrated safely to new formats and software and hardware environments.

• Standardized interoperability rules can be applied effectively, making it possible to interface dispersed information systems, reuse information and develop cross- organizational services.

• Information loss can be minimized.

What are the challenges for the future?

As countries in Africa and across the world work toward maximizing new opportunities for sharing, coordinating and reusing information, they need to be able to rely on high-quality digital records as an essential evidence base for sustainable development. Countries that do not develop requirements for managing digital records will be at a significant disadvantage.

In the new information environment, laws, administrative placement and responsibilities for digital records management need to be reviewed, broadened and extended to support changing requirements for using information – whatever its form – as evidence. The longstanding role of national archives, as the agencies with statutory responsibility and expertise in managing and protecting records, will remain essential, but it will need to be expanded and harmonized with the role of agencies responsible for digital governance and openness.

Written by

Anne Thurston has worked with governments in lower resource countries for over four decades to share solutions for managing public sector records. She lived in Kenya between 1970 and 1980, where she conducted research and worked for the Government’s Kenya National Archives. In 1980 she became a lecturer, later a Reader, in International Records Studies at University College London. She established the International Records Management Trust in 1989 and, retaining the title of Director, she has concentrated fully on the work of the Trust since 1996.