Loading...
Important Notice - Forums is archived

To simplify things and help our users to be more productive, we have archived the current forum and focus our efforts on helping developers on Stack Overflow. You can post new questions on Stack Overflow or join our Discord channel.

Product icon
TUTORIAL

Vaadin lets you build secure, UX-first PWAs entirely in Java.
Free ebook & tutorial.

Rich web apps and search engines like a Google - indexing problems and tri

Dmitri Livotov
1 decade ago Dec 18, 2009 5:36pm
Dmitri Livotov
1 decade ago Jan 01, 2010 2:33pm
Jean-François Lamy
1 decade ago Jan 01, 2010 4:10pm

It is trickier than that. If you page is made up of several components which can change independently, and if you have many ways to get to the same page, etc. You can end up with a total mess. You have no control over what will be retrieved.

We ended up creating a crawler, and giving it specific rules as to what to crawl and where not to go. The crawler created a shadow site, and the shadow site was being indexed.

Dmitri Livotov
1 decade ago Jan 01, 2010 4:19pm
Jean-François Lamy
1 decade ago Jan 01, 2010 7:24pm
Dmitri Livotov
1 decade ago Jan 02, 2010 2:43pm
Jean-François Lamy
1 decade ago Jan 02, 2010 6:39pm
Marc Englund
1 decade ago Jan 05, 2010 9:22am
Dmitri Livotov
1 decade ago Jan 05, 2010 8:42pm
Steve Jonathan
1 decade ago Apr 15, 2010 11:29am
Nicolas Brasseur
1 decade ago Jun 18, 2010 8:45am
Dmitri Livotov
1 decade ago Jun 18, 2010 8:49am
Joe Jaro
7 years ago Jul 09, 2015 6:31pm