How Action Learning Institute automated their training data

Action Learning Institute, an Australian registered training organisation, needed a way to keep their MyLearningMap application up to date with the latest training units from training.gov.au. Here’s how they solved it:

The challenge

Action Learning Institute wanted to bring official training certifications and qualifications from training.gov.au into their existing web application, MyLearningMap. But there were a few hurdles:

• Unit descriptions on training.gov.au are written for humans, not machines. The text formatting is inconsistent and unstructured.
• MyLearningMap needs specific, detailed criteria, skills and requirements that are buried within that text.
• Training units can change at any time, so the data needed to stay fresh (no more than a day old).

The solution

An automated system now runs every morning to keep everything in sync. Here’s what happens:

1. Check for updates – The system connects to the training.gov.au SOAP API to find out which unit descriptions have changed since yesterday.
2. Scrape and structure the data – A web scraper extracts the criteria, skills and requirements from the updated units, then converts the messy text into clean, structured database records.
3. Import into MyLearningMap – The structured data flows into the MyLearningMap Training Unit Database, ready to use immediately.

The result

The system runs automatically each day, so Action Learning Institute’s team doesn’t need to manually check for updates or copy data across. Their application always has the latest training information.

More importantly, users of MyLearningMap can now map official training metrics against evidence that their training participants are competent in each area. The data is structured in a way that makes this possible, even though it started as inconsistent text.

Action Learning Institute now has a practical solution that saves their team time and keeps their training records accurate and current.

Need help making your systems work better? Get in touch with our team to discuss your project here.