Setting axis limits in ggplot has behaviour that may be unexpected: any data that falls outside of the limits is ignored, instead of just being hidden. This means that if you apply a statistic or calculation on the data, like plotting a box and whiskers plot, the result will only be based on the data within the limits.
In other words, ggplot doesn’t “zoom in” on a part of your plot when you apply an axis limit, it recalculates a new plot with the restricted data.
For example, here is a boxplot without any axis limits:
and here is the same one with a set Y axis limit:
Note how the box and whiskers are recalculated in this plot. To be fair, ggplot does warn you that it is removing rows!
If you want to zoom in without removing values, use coord_cartesian: