A manual analysis was conducted on 2,364 sampled questions, leading to the development of a taxonomy of challenges that includes 27 distinct categories, such as:

- Prompt Design, - Integration with Custom Applications, and - Token Limitation.

Based on this taxonomy, the study also summarizes key findings and provides actionable implications for LLM stakeholders, including developers and providers.

The Six Challenges Working With LLMs

1. Automating Task Processing: LLMs can automate tasks like text generation and image recognition, unlike traditional software that requires manual coding.

2. Dealing with Uncertainty: LLMs produce variable and sometimes unpredictable outputs, requiring developers to manage this uncertainty.

3. Handling Large-Scale Datasets: Developing LLMs involves managing large datasets, necessitating expertise in data preprocessing and resource efficiency.

4. Data Privacy and Security: LLMs require extensive data for training, raising concerns about ensuring user data privacy and security.

5. Performance Optimisation: Optimising LLM performance, particularly in output accuracy, differs from traditional software optimisation.

6. Interpreting Model Outputs: Understanding and ensuring the reliability of LLM outputs can be complex and context-dependent.