Research: V100 Volta arch 70 support

[manu69x]

Research Stage

• Background Research (Let's try to avoid reinventing the wheel)
• Hypothesis Formed (How do you think this will work and it's effect?)
• Strategy / Implementation Forming
• Analysis of results
• Debrief / Documentation (So people in the future can learn from us)

Previous existing literature and research

I don't know if it can be useful but I wanted to report that while with version v0.4.2 the V100 arch 70 GPU did not work, now with version v0.4.3 preview it works without errors, but very slowly. A test on 128k ctx that with turboquant turbo3 takes 253 seconds with bellama kwarn3 takes 11480 seconds.

Hypothesis

No response

Implementation

No response

Analysis

No response

Relevant log output