Ensembling Large Language Models for Human Activity Recognition in Smart Environments

Summary

Recognizing human activities accurately is crucial in smart environments, supporting applications in healthcare, security, and automation. Traditional machine learning struggles with the complexity and variety of human behaviours recorded by sensors. This project aims to improve Human Activity Recognition (HAR) by using “ensemble” methods, which combine multiple models to strengthen results. Specifically, it leverages Large Language Models (LLMs)—which excel at understanding sequences and can work effectively with limited labelled data.

Recent advances in LLM-based HAR systems, such as LLaSA and HARGPT, have shown encouraging results. These models bring a “zero-shot” capability, meaning they can recognize new activities without specific examples, a key benefit in HAR, where labelled data is often scarce.

This project will combine different ensemble methods, like voting or stacking, to handle data from various sensors (such as wearable and environmental). By integrating LLMs like GPT or BERT as base models, the approach reduces the need for manually creating features from data, which is typically labour-intensive. Using LLMs in this way has challenges, such as high computational demands, but techniques like transfer learning (where models are trained on related tasks) and tuning will help make these models more practical.

The research will use existing datasets to train and test the models, comparing their accuracy, efficiency, and adaptability. Expected contributions include creating more accurate and flexible HAR tools that can support healthcare, security, and energy management across diverse settings, advancing the way smart environments respond to human activity.

References:
1. Cleland, I., et al. (2024). Large Language Models for Activity Recognition in Smart Environments.
2. Wu, S. et al. (2023). A geometric framework for multiclass ensemble classifiers.
3. Imran, S. A., et al. (2024). LLaSA: Large Multimodal Agent for Human Activity Analysis.
4. Ji, S., et al. (2024). HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?

The School of Computing at Ulster University holds Athena Swan Bronze Award since 2016 and is committed to promote and advance gender equality in Higher Education. We particularly welcome female applicants, as they are under-represented within the School.

Essential criteria

Applicants should hold, or expect to obtain, a First or Upper Second Class Honours Degree in a subject relevant to the proposed area of study.

We may also consider applications from those who hold equivalent qualifications, for example, a Lower Second Class Honours Degree plus a Master’s Degree with Distinction.

In exceptional circumstances, the University may consider a portfolio of evidence from applicants who have appropriate professional experience which is equivalent to the learning outcomes of an Honours degree in lieu of academic qualifications.

• Sound understanding of subject area as evidenced by a comprehensive research proposal
• A demonstrable interest in the research area associated with the studentship

Desirable Criteria

If the University receives a large number of applicants for the project, the following desirable criteria may be applied to shortlist applicants for interview.

• First Class Honours (1st) Degree
• Masters at 70%
• For VCRS Awards, Masters at 75%
• Experience using research methods or other approaches relevant to the subject domain
• Work experience relevant to the proposed project
• Publications - peer-reviewed
• Experience of presentation of research findings

Equal Opportunities

The University is an equal opportunities employer and welcomes applicants from all sections of the community, particularly from those with disabilities.

Appointment will be made on merit.

Funding and eligibility

This project is funded by:

• Department for the Economy (DfE)
• Vice Chancellor's Research Scholarship (VCRS)

Our fully funded PhD scholarships will cover tuition fees and provide a maintenance allowance of £19,237 (tbc) per annum for three years (subject to satisfactory academic performance).  A Research Training Support Grant (RTSG) of £900 per annum is also available.

These scholarships, funded via the Department for the Economy (DfE) and the Vice Chancellor’s Research Scholarships (VCRS), are open to applicants worldwide, regardless of residency or domicile.

Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Due consideration should be given to financing your studies.