# Eddie's Learning Record 25

**Published by:** [Eddie's Learning Records](https://paragraph.com/@eddiehe/)
**Published on:** 2022-11-29
**URL:** https://paragraph.com/@eddiehe/eddies-learning-record-25

## Content

1. DurationMonday, November 21st, 2022 - Saturday, November 26th, 2022 2. Learning Record2.1 Fine-tuned ModelsWhen images have only two classes which means the labels are "0" and "1", the `label_mode` of `tf.keras.utils.image_dataset_from_directory` should be `binary` and the loss_fn should be `BinaryCrossentropy` instead of `SparseCategoricalCrossentropy`. Otherwise, the accuracy will jitter around 50%, which means the model learns nothing. 2.2 Learned Swin TransformerI watched the paper [1] and read the code. It took me a few days to understand the `Window-based Self-Attention & Shifted Window-based Self-Attention` and the `Swin Transformer Block`. 2.3 Refactored the CodeI built a py file to store all functions for loading the datasets. Consequently, the code could work only by changing the basee_dir of the dataset. 3. Feeling3.1 GladI was glad that the models ran well. 3.2 PerplexedThe models seemed to generate relatively good results on some small datasets. But they didn't work that well on the micro-expression dataset. Besides ViT, SL-ViT, Swin Transformer, I also found that there are lots of other types of transformer models. It seems impossible to learn all of them.

## Publication Information

- [Eddie's Learning Records](https://paragraph.com/@eddiehe/): Publication homepage
- [All Posts](https://paragraph.com/@eddiehe/): More posts from this publication
- [RSS Feed](https://api.paragraph.com/blogs/rss/@eddiehe): Subscribe to updates
- [Twitter](https://twitter.com/eddiehe99): Follow on Twitter

## Optional

- [Collect as NFT](https://paragraph.com/@eddiehe/eddies-learning-record-25): Support the author by collecting this post
- [View Collectors](https://paragraph.com/@eddiehe/eddies-learning-record-25/collectors): See who has collected this post