diff options
author | Tim Dettmers <TimDettmers@users.noreply.github.com> | 2022-09-05 16:29:25 -0700 |
---|---|---|
committer | GitHub <noreply@github.com> | 2022-09-05 16:29:25 -0700 |
commit | aca55881b9815a462142f42f3ff0dc917830d85c (patch) | |
tree | 75477acfbce2da2a753ee21d4cf0da64f3f50ea5 /README.md | |
parent | 92a3363096e10ad6a5c4e944af898bd1186d806a (diff) | |
parent | eab4d8232d558f2e6bd7f7cc3d00e2e6e94f4e80 (diff) |
Merge branch 'main' into remove_unused_code
Diffstat (limited to 'README.md')
-rw-r--r-- | README.md | 14 |
1 files changed, 8 insertions, 6 deletions
@@ -23,12 +23,12 @@ Resources: 1. Comment out torch.nn.Linear: ``#linear = torch.nn.Linear(...)`` 2. Add bnb 8-bit linear light module: ``linear = bnb.nn.Linear8bitLt(...)`` (base arguments stay the same) 3. There are two modes: - - Mixed 8-bit training with 16-bit main weights. Pass the argument ``use_fp16_weights=True`` (default) - - Int8 inference. Pass the argument ``use_fp16_weights=False`` + - Mixed 8-bit training with 16-bit main weights. Pass the argument ``has_fp16_weights=True`` (default) + - Int8 inference. Pass the argument ``has_fp16_weights=False`` 4. To use the full LLM.int8() method, use the ``threshold=k`` argument. We recommend ``k=6.0``. ```python # LLM.int8() -linear = bnb.nn.Linear8bitLt(dim1, dim2, bias=True, use_fp16_weights=False, threshold=6.0) +linear = bnb.nn.Linear8bitLt(dim1, dim2, bias=True, has_fp16_weights=False, threshold=6.0) # inputs need to be fp16 out = linear(x.to(torch.float16)) ``` @@ -115,7 +115,8 @@ We thank Fabio Cannizzo for his work on [FastBinarySearch](https://github.com/fa ## How to cite us If you found this library and found LLM.int8() useful, please consider citing our work: -``` + +```bibtex @article{dettmers2022llmint8, title={LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale}, author={Dettmers, Tim and Lewis, Mike and Belkada, Younes and Zettlemoyer, Luke}, @@ -124,8 +125,9 @@ If you found this library and found LLM.int8() useful, please consider citing ou } ``` -For 8-bit optimizers or quantization routines please consider citing the following work. -``` +For 8-bit optimizers or quantization routines, please consider citing the following work: + +```bibtex @article{dettmers2022optimizers, title={8-bit Optimizers via Block-wise Quantization}, author={Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke}, |