Providing real-time AI queries at the end-user level is exponentially expensive.
The current AI Assistant feature currently offers task-based responses by chapter.
Effectively, we can pre-store responses generated from static prompts / task before the application build.
While pre-building these responses represents a one-time investment with initial time and cost risks, it proves sustainably cost-effective in the long run.
For instance, storing 3 varied responses for each of 1189 chapters / tasks requires approximately 3,567 API requests.
Based on typical pricing per token, this has a calculable cost (U$ ???).
If a model permits 50 requests per minute, generating these would take around 72 minutes.
Prior to implementing this, the SAB team would need to conduct research to determine the estimated pricing based on the chosen model and AI provider.