I can hear the concern in your double question marks, but there is no need to worry. :) In all honesty, when I have a weekend off from the racing world, I like to kick it on campus.
There are tons of people who have commitments outside of their love of 'shredding the gnar.' For example, my friends, Kate, Sarah, and Teresa, will go ski on Saturday, but be around Sunday because of an event with GreenUpMC or SGAC. Even die hard skiers like JBo, a student who skied 87 days this year, are in the Student Association and Founder's Society. The moral of that story is that there are ALWAYS people on campus not skiing/riding.
There are also a lot of people who flat-out have never skied or boarded in their lives. (There are also those like my friend, Meg, who 'strictly tube' as she likes to say.) If it is something you want to learn, the Wilderness Program put on Learn to Ski and Snowboard programs. My opinion is that you are in Vermont and even you are a Floridian, you should ski or ride once during your four years because its part of the SMC life. (Also note that my suitemate who is born-and-raised VT has never been skiing.)
I can also promise you that there is something to do on-campus or in Burlington if you are not at the mountain, so you will never be bored!
Hope this helps! I wanted to cover a lot of bases!