I never found college to be too difficult, once I was doing what I wanted to be doing.
Now, my first year was in a pre-med program, and about 3 months in I decided I hated chemistry in all it's forms. Then, college was the most difficult thing I've ever done.
But my 2nd school I went to was a lot "easier" even if there was more work, because I liked what I was doing. That makes a big difference.