Scrutiny over promises – researchers urge caution over the promises of AI to ‘fix’ aged care

As with countless technologies and leaps of the past, there is usually an excited flocking to adopt the new. This has monumentally been the case with AI, as most sectors have seemingly embraced the technology with gusto. Yet a team of Australian expert researchers have committed resources to methodically look beyond the hype. The team researched how companies are not only packaging AI for aged care, as a promise to ‘fix’ all of its problems, but that this messaging has caused “distraction” from serious structural flaws in the system, as well as reinforcing deeply harmful sentiments of ageism. When approaching the conversation as to what place AI has to play in aged care, the researchers offer that caution and scrutiny must be used by leaders, advocates and the populace in navigating what aged care needs at its heart. While many who call aged care their life-long vocation would love a cure-all, others know that believing in one could bring considerable harm to those that need the sector most. Aged care experts and leaders of experience, not trend, know that to correct to a robust, realistic and sustainable future for human-beings, continued cross-sector campaigning and work is needed to tackle the deep systemic issues. 

Wider context

Dr Barbara Neves, Alexandra Sanders, Dr Geoffrey Mead, Dr Naseem Ahmadpour, Professor Alex Broom and Dr Kalervo Gulson are academics out of the University of Sydney University with decades of research years between them. In an important pivot to gauge what is happening in aged care in the newest tech revolution, these academics entered into an extensive examination of tech products and promises. What they found, they considered concerning.

The researchers recognised that what the Royal Commission identified, alongside many of those who have spent years pouring themselves into aged care experience, was a flawed system. In the wake of the Commission being handed down however, the team started to notice technology companies promising that artificial intelligence (AI) would be the vehicle by which all flaws would be solved, from healing senior loneliness to staff shortages.

They explain that this new trend has been coined “agetech”, an industry and push that is likely to rise to a value of A$170 billion by 2030. Yet what they, professionally and ethically, were troubled by was how agetech’s ‘fixes’ were distracting from deeper systemic flaws. Conveying solutions only to be found in AI, they assessed, undermined the changes aged care needs, that seniors, staff and many sincere provider heads are already working towards but need government attention and resources for.

Shaping narrative

In an extensive new study, these researchers, examined 33 agetech companies offering AI products for aged care across Australia, East Asia, Europe and North America, and how these products were marketed, particularly when it came to products of monitoring tools and companion robots.

The team found that across all the websites, promotional messaging and product descriptions, aged care was only shaped as inefficient, understaffed and incapable of rising to meet a growing ageing population. Seniors were described as only extremely frail and too populous. Care staff were painted as only overstretched, with the dominant language leading to a conclusion, human care as being inherently flawed.

Invariably, all the companies and products led to a final offering, AI was the answer. The research team, in seeing a united and growing trend of messaging and not only that, adoption as the agetech industry continues to grow has called for caution and understanding. Particularly as the Australian government increasingly shows a willingness to adopt AI and pay for it with public funds, there is signalling that many are subscribing to this framework of only technological rescue.

The researchers highlight that their findings show a dominant singular narrative has been distracting from “structural problems”, also leading to other consequences such as further entrenching damaging sentiments of ageism. As Australia and the sector grapple to land the new age care act, committing to a focus on dignity and autonomy of humans, the researchers advocate that before AI is accepted as the only cure, it is imperative for leaders to scrutinise and understand what is being sold.

The cure

The team found that the companies promoting AI claim that the software will “predict falls before they happen” and identify complex health changes that humans will miss. The AI products promised a removal of incompetence and provided “unprecedented” rises in quality and safety.

The experts found that the language used in all the product marketing was inspiring and “revolutionary”. They also found that across all the products a trend emerged, aged care was routinely shaped as only a failure of efficiency and public delivery.

In the imagery, seniors were shown as sitting passively, with the greatest struggles being how to handle a mobility aid. More concerningly, most seniors were only shown in part, a body part here or there, attached to monitoring devices. The researchers saw that seniors were expressed as statistics, how often they fell, malnutrition rates and how likely they were to end up in hospital.

The researchers were alarmed as the trend intensified. According to the AI companies, seniors were events waiting to occur, sources of data to be mined. A company clearly marketed that through their AI tech, providers could turn intimate daily needs into reputational clout surrounding “optimal care”, by turning showering into a “trackable metric”.

In a further twist that front-line staff and many provider heads have seen as profoundly disrespectful, AI companies indicated that front-line staff’s labour was “time-consuming” and “error-prone”. The AI messaging consistently shaped care workers as a liability problem, and AI as the solution. Care workers were only seen as well-intentioned but inherently always unreliable, needing technological monitoring. Many companies the team found, marketed products that would track the movement of staff, and automatically ping managers if anyone was held up, whatever the reason.

Techno-solutionism

The team go on to explain the wider narrative AI companies and agetech are attempting to shape. The researchers have seen a rise of agetech companies selling their products right after significant pushes to see the aged care sector as only broken, overwhelmed by ballooning costs, inefficiencies, and without competent or passionate professionals to steer towards solutions.

It is within this ‘all hope is lost’ narrative that the researchers have found AI companies enter in with their solutions, 24/7 tracking of everything, crystal-globe-like predictive algorithms that almost seem to promise alerting right before a fall. They are only presented as superior, perfect and objective.

The team explains this is a clear example of what has been growing for years, “techno-solutionism”, where social issues are configured so as to conclude that only technical and technological solutions are the answer.

The team are impassioned and evidenced, it is critical that provider leadership, sector experts and front-line staff are uniformly aware that AI is far from “neutral”. The inherent data sets that were and are used to train AI software functioning have been consistently proven to derive from datasets that exclude people who are older, and utilise and respect people that are younger, healthier and follow educational and cultural norms.

Academics, researchers from across the globe are joining a growing cohort that is calling for AI implementation to be done with significant care, scrutiny and caution. The team in their research combed copious studies that have found that AI, in its inherent design and eventual implementation, can be based on stereotypical and damaging narrow data, whereby seniors are only technophobic, passive and frail.

Researchers from the University of New South Wales have also found that data sets for AI hiring software have baked into it ageist, classist and racist understandings of labour value. AI tools have been found to reject candidates with unusual labour history. And in so doing, thousands of passionate and hard-working people may have already been prevented from entering into aged care to be a part of the push for quality and dignity-based care.

AI is not the silver bullet

The researchers are clear, while immense investment has meant eye-watering marketing outlays to shape the problem of aged care and how AI will fix it, digging deeper is what the aged care sector deserves and needs.

Echoing the insight of aged care experts and advocates of decades, the aged care crisis stems from many different sources, decades of social and political choices that have shaped how important care and ageing is, and particularly how it’s managed and funded. Pervasive systemic flaws, institutionalism, regulatory burden and failures, cost-cutting, as well as a society that fears and distances itself from ageing, all of this, the team say, has gone into the state of aged care today.

The researchers have found that AI companies have shaped the crisis as perfectly being solved by technical solutions, rather than social or political reform. And while that seems easiest, therein, they say, perhaps lies the first inkling that something is wrong with that approach. 

This easy route they assess, distracts from the fact and need for broader and more difficult reforms. Many of the most important elements in people’s lives, from cherished careers, to partners and children are far from easy, leading psychologists attest, it is in the commitment to persevere and work, to passionately show up day after day that lasting progress is made. The easy fix usually doesn’t deliver. 

For aged care professionals who have committed decades to bringing their professional and personal conviction to aged care, the promise of AI has not rung true. Seniors and the sector deserve more than a silver bullet. The sector deserves sustainable and stellar excellence, shaped by the people that do the work day in, day out.

The researchers have seen that the AI marketing says that its systems will eliminate the work of aged care. But not only does this miss the point of thousands of those in aged care, that far from ‘work’ it is a willing and joyful calling, where hard work is done without rose-tinted glasses from a deeper place of care and conviction, it also misses that in many cases, AI has loaded up staff with work, distracting from care.

More work

For provider leadership it is worthwhile to note that the research team found that AI systems have resulted in significant human labour requirements due to AI implementation. These adoptions oftentimes creating more work than they promised to alleviate.

The findings showed that front-line staff not only had to learn new systems, make sense of copious data, they were also constantly distracted by notifications and false alarms. When providers implemented new technologies, staff were tasked with further managing technology that needed consistent management in calibration and maintenance, taking further time away from the floor and care of seniors.

The team looked at extensive studies on what this did to staff morale and the implications for the care they were able to provide. They discovered that global research indicated that AI adoption had resulted in rising worker staff numbers, staff needing to triage care responsibilities alongside tech errors and troubleshooting, all within their original limited time and limited training.

Concerningly, as advocates have also attested, as care time is being loaded with technological wrestling, the vital relationship elements of care, building trust and relationships, being able to discern subtleties in mood and demeanour, these are further squeezed out in time and focus priority. The results, both advocates and the researchers say, is that the ability to provide quality care is further eroded.

The irony, in many of the cases of AI adoption in aged care has been that seniors have had care quality deteriorate. The team found that when care was orientated to efficiency metrics and cost management alone, seniors were shaped as issues to be navigated rather than people with an eclectic array of histories, preferences, needs and importantly, capabilities and resources to give.

No single solution

The researchers acknowledge that aged care in Australia has significant fault-lines that need to be addressed. Reform must be made by government and sector leaders, alongside advocates and seniors, to pivot systems and humans working to what is sustainable and dynamic. They agree that the solutions will need to be varied and diverse but insist that most of them will have nothing to do with AI.

The researchers encourage staff ratios to be re-examined by government, to fund and underpin meaningful conversations of humans to humans. This method has been decidedly found by neuroscientists to be the most effective way to tackle loneliness.

At a greater level the researchers call for wages that reflect the value and complexity of the work. Analysts hedge this will require greater governmental reform to ensure that the tax-burdens to pay for this increase are spread across income and company tax bases.

The team also encourage government reform to facilitate aged care to include community-based models. Provider leadership are already calling for support to innovate care through intergenerational models, where toddlers, teens and all in-between can receive and give care as needed. Funding and reform must meet this dynamic and impactful solution.

The researchers note that at best, AI can be used as a supporting tool for care practices but it is vital that staff are given the time and training to use it well, and particularly, turn off incessant notifications that distract from the critical human care. Senior and staff empowerment, they advocate, must be the central to the functioning of aged care.

Provider leadership has already been vocal, that in investing in their care staff to build relationships with residents in facilities, not only has morale lifted for all, staff turnover has diminished, well-being increased and operational costs reduced.

The researchers are clear about the seriousness of this situation; they articulate that if AI companies are permitted to be the only voices to define what is broken, the sector could allow a narrow idea of what solutions look like. They see this as potentially meaning certain sectors thrive, such as tech while consequences are entrenched in “far less caring and humane” aged care.

There is significant work of reform yet ahead for aged care, advocates and leaders say. For the many who have committed to improving aged care in their roles, they continue to advocate, and while it is difficult, it is worthwhile. For the seniors at the heart of aged care, sincere and passionate human professionals, leaders and front-line staff, will show up each day to do the work and calling of aged care.