github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-12 12:47:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Fix] resolve conflicts of rebasing feat/speculative-decoding (#5557)

Browse Source 
- resolve conflicts of rebasing feat/speculative-decoding
This commit is contained in:
Yuanheng Zhao
2024-04-07 14:53:30 +08:00
committed by ocd_with_naming
parent e1acb58423
commit e60d430cf5
6 changed files with 47 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
colossalai/inference/batch_bucket.py
View File

@@ -97,7 +97,6 @@ class BatchBucket:
@property
def num_tokens_to_verify(self) -> int:
assert self.use_spec_dec and self._num_tokens_to_verify is not None
return self._num_tokens_to_verify
def set_use_spec_dec(self, num_tokens_to_verify: int = 5) -> None: