Add accuracy benchmarks for llama 3

[vkkhare]

Post-training predictors on llama 3 output, we need to evaluate

• MMLU and AIME, refer to further benchmarks
• Under int8 quant, fp16 and fp32
• With and without explicitly sparsifying LLMs like Prosparse

[void-memories]

Waiting on 10% LORA results
Waiting on profiling results

to decide next steps.

[kaselby]

10% lora results with sparse predictors on layers 5-31 are essentially identical to ground truth. I am running the profiling now.

[nrjpoddar]

@vkkhare to implement KV caching