Prompt Tuned Embedding Classification for Industry Sector Allocation

We introduce Prompt Tuned Embedding Classification (PTEC) for classifying companies within an investment firm’s proprietary industry taxonomy, supporting their thematic investment strategy. PTEC assigns companies to the sectors they primarily operate in, conceptualizing this process as a multi-label text classification task. Prompt Tuning, usually deployed as a text-to-text (T2T) classification approach, ensures low computational cost while maintaining high task performance. However, T2T classification has limitations on multi-label tasks due to the generation of non-existing labels, permutation invariance of the label sequence, and a lack of confidence scores. PTEC addresses these limitations by utilizing a classification head in place of the Large Language Models (LLMs) language head. PTEC surpasses both baselines and human performance while lowering computational demands. This indicates the continuing need to adapt state-of-the-art methods to domain-specific tasks, even in the era of LLMs with strong generalization abilities. The proposed method integrates constrained decoding using Trie Search and jointly optimizes a soft prompt along with the classification head, demonstrating improved scalability, efficiency, and classification performance on proprietary and public datasets. Our contributions include adapting Trie Search to prevent repetitive label prediction, introducing PTEC with differential learning rates, and providing empirical evidence that PTEC's performance generalizes well across datasets with varying pretraining knowledge.