Allow search engines to crawl and index blog

How would we set up our forum and blog to be crawled and indexed by search engines?

Is it as easy as adding an allow path to our robots.txt file? If so, which path would we use?

Mike

You don't have to do anything special. It'll already crawl through Joomla. Use Google webmaster tools to test the crawler against your site and adjust as necessary. There's hundreds of tools around the net for testing crawler efficiency, please search (Google, not here as we've none).

krileon wrote: You don't have to do anything special. It'll already crawl through Joomla. Use Google webmaster tools to test the crawler against your site and adjust as necessary. There's hundreds of tools around the net for testing crawler efficiency, please search (Google, not here as we've none).

Kirleon, will the crawlers search the blogs, forum, and profile information within restricted areas (must be logged in) and within the Tabbed jQuery AJAX content?

Do you think Joomlapolis' forums, blogs and profile wall are searchable? I would think so if you don't have to do anything special. but I find any of myy of my forum or blog entries from this site on Google?

I've tested my site and the AJAX content (tabbed blogs, forums, about me, and profile wall does not show up using the Fetch as Google. In other words, Google can not see the hidden ajax content within the tabs so it does not index it. I'm thinking that is why I can't find any of my user profile content on this website either.

Has anyone figured out and successfully configured their web servers to allow the community builder forums, blogs, and profile wall to be indexed by Google?

For example, using this Google tutorial?
developers.google.com/webmasters/ajax-crawling/

Mike

Basically, I want to offer my members with their own forums and blogs that are search engine friendly so they can publish on our site and can be found via Google and Bing.

If someone can point me in the right direction I'd appreciate it.

They'll crawl anything that's publicly available. You can't have your sites information public and private at the same time. That's not how it works. If you want your information crawled then you need to make it public.

Sorry, I don't know what more to tell you. I'm no SEO expert. You really should hire an SEO expert to help in this regard.

Profile tabs are not AJAX. The information is all in the pages source. Its display is simply disabled. As you click through the tabs it shows another then hides the other.

Thank you Krileon,

Although the content is delivered to the page and is hidden, Google does not crawl it. Probably because of what you say...it's private by nature.

Can you direct me how to make a profile public?

Shaka Brah,
Mike
www.KillerSurflinks.com

Profile access can be configured within CB > Configuration > User Profile then set "Allow Access To:" to "Everybody".