Books that are out of copyright, pretty much fair game.
Books that are in copyright, the law is vague. The principle is "fair use." If your web site is free and is "celebrating" the books, you can probably use an image or two from each and some quoted text safely. The "rule of thumb" in academia is 10%. Of course unless your web site becomes wildly popular you are unlikely to attract the notice of anyone who would sue you, and the most you would likely risk is a lawyerly demand to take the site down.
Stanford has a pretty good site: [link]
Here's the text from the government about fair use: [link]