Don't fret over the scale. Assuming its accurate, assuming its consistent (if inaccurate), your body is going to fluctuate over the day(s). If you use a scale pick one day and one time of day and focus on that number and how it changes. Best time in the morning after you take a dump.
And just look in the mirror.