diff options
| author | Louis Del Valle <92354925+nero-dv@users.noreply.github.com> | 2023-05-10 22:05:18 -0500 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-05-10 22:05:18 -0500 |
| commit | c8732dfa6f763332962d97ff040af156e24a9e62 (patch) | |
| tree | 92723cd92da2d1557571778b1c44c81182eb8ea4 /extensions-builtin/prompt-bracket-checker | |
| parent | 8aa87c564a79965013715d56a5f90d2a34d5d6ee (diff) | |
Update sub_quadratic_attention.py
1. Determine the number of query chunks.
2. Calculate the final shape of the res tensor.
3. Initialize the tensor with the calculated shape and dtype, (same dtype as the input tensors, usually)
Can initialize the tensor as a zero-filled tensor with the correct shape and dtype, then compute the attention scores for each query chunk and fill the corresponding slice of tensor.
Diffstat (limited to 'extensions-builtin/prompt-bracket-checker')
0 files changed, 0 insertions, 0 deletions