Speaker
Description
Theme:
Generative AI (GenAI) and Machine Learning (ML) in statistical production.
Abstract:
This study outlines a practical initiative by the Department of Statistics, Malaysia
(DOSM) with the purpose of developing a responsible, low-risk pathway for exploring
Artificial Intelligence in official statistics. Our goal was to understand how AI, when
firmly guided by established statistical standards, could be integrated as a support tool
to enhance productivity, internal analysis, and data coherence without compromising
the quality of our official outputs.
Our methods centered on a dual-track, standards-first approach. We conducted
controlled practises in a secure environment, training officers in prompt engineering to
use Generative AI for drafting standard metadata from existing templates. In parallel,
we explored the application of basic Machine Learning models for internal analytical
support, such as generating forecast scenarios and nowcasts to inform our production
planning and quality assessment processes. Importantly, these ML outputs are treated
as supplementary analytical aids and not as official statistical releases. Alongside this,
we initiated foundational work to systematically review, clean, and document our core
data processes and structures, aligning them with SDMX-like standards to create a
reliable basis for future automation. The key findings from this exploration were
definitive: AI use significantly reduced time spent on initial drafting for routine tasks,
firmly establishing its role as an assistant. We found that an officer's ability to craft
precise, standards-specific prompts is a critical new skill. Furthermore, the pilot use of
ML for internal forecasting proved valuable for cross-checking trends and identifying
anomalies in ongoing data collection, thereby enhancing our validation processes.
Crucially, the quality and reliability of all AI and ML outputs were directly tied to the
rigidity of our input standards and data governance.
In conclusion, this initiative confirms that the immediate value of AI for a National
Statistical Office lies in two areas: responsibly accelerating rule-based administrative
tasks and providing sophisticated analytical support for internal quality assurance and
planning. The initiative underscores that statistical and metadata standards are the
non-negotiable foundation for any successful AI integration, ensuring these tools
support rather than subvert official processes. DOSM’s experience demonstrates that
investing in prompt engineering skills, exploring ML for auxiliary analysis, and
strengthening core data governance are essential first steps, providing a replicable
model for building the trust and readiness required for the future of statistical
production.
