flash attn
This commit is contained in:
parent
eae0f04b8e
commit
7d78ebec79
4 changed files with 58 additions and 16 deletions
|
|
@ -40,6 +40,12 @@ pip install open-mythos
|
|||
#uv pip install open-mythos
|
||||
```
|
||||
|
||||
To enable Flash Attention 2 in `GQAttention` (requires CUDA and build tools):
|
||||
|
||||
```bash
|
||||
pip install open-mythos[flash]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue