Recently I have been setting up blogs in my Joomla sites using core features and a few add-ons (non-commercial of course). Here are my steps...

Initial Setup:

1. Create a Section called Blog
2. Create each blog category wanted and assign it to the Blog section
3. Create your menu item (I call mine Blog) and utilize the Articles->Section Blog Layout type
4. Select your Blog section from the Section dropdown
5. Set your Basic Parameters (for this example, I used... 0 leading articles, 15 intro articles, 1 column, 0 links)
6. Set your Advanced Parameters (for this example, I used... Most recent first primary order, etc)
7. Set your Component Parameters (this is where you would set the author's name to show, show the hits, show the category filed under, make the title be linkable, etc)

Blog Modules:

1. For a "Blog Categories" module, I installed a third-party module (the mod_categories module) that displays a bulleted list of all the categories assigned to a certain section. The module also shows a counter for the number of articles in each category. The third-party module is here: http://extensions.joomla.org/extensions/structure-a-navigation/sections-a-categories/7489
2. For a "Blog Roll" module, I installed a third-party module that displays Web Links from a certain Web Links category. The third-party module is here: http://extensions.joomla.org/extensions/directory-a-documentation/weblinks/1979
3. For a "Latest Posts" module, I simply used the core mod_latestnews module
4. For a "Blog Calendar" module, I tried to give the following third-party module a try, but it had advertising that if removed made the module not function correctly. I still need to re-code it to make it brand/ad free. http://extensions.joomla.org/extensions/news-display/articles-listing/6305

Blog Comments:

For this, I utilized the third-party component. I tried out Azrul's JomComment at first (commercial component), but then switched over to a non-commercial add-on. Here you can find a bunch of them: http://extensions.joomla.org/extensions/contacts-&-feedback/articles-comments

Blog Users:

Managed using the core Joomla user manager of course :)