[Meta] Older blogoscoped forum posts missing from Google?Niraj Sanghvi ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | Thursday, February 21, 2008 17 years ago • 4,862 views |
I was trying to track down a post to respond to Rohit's post about Adsense and Google Accounts merging (http://blogoscoped.com/forum/124219.html) since I knew I had posted something a few months ago with almost the same screenshot. And even though I remembered some of the wording of the post, I couldn't track it down via either the search on blogoscoped or on Google.
Finally I remembered I had taken a screenshot, and I tracked down the image and got the date. I paged through the forum all the way back to that date, and sure enough, there was my post.
I then realized that it looks like forum posts that are more than a few months old are not searchable, even when using their exact titles and site:blogoscoped to find them. But recent posts appear just fine. I'm not sure where the boundary is, but I suspect that either the threads created before the move to blogoscoped.com are somehow not showing up, or that posts past a certain age are not showing up.
Of course, I've only tried a handful of examples, but I'd be interested to know if others are finding the same thing. There's been a lot of great discussions and it'd be unfortunate if they're not searchable in any way. |
Niraj Sanghvi ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Update: It looks like Ask and Yahoo have the posts correctly, but Google does not. In fact, searching for a thread title revealed two blogs that had cached the thread and even linked to it, but the blogoscoped.com/blog.outer-court.com original is not appearing in the result:
http://www.google.com/search?q=%22adsense+account+merge%22&complete=1&hl=en&safe=off&filter=0
http://search.yahoo.com/search;_ylt=A0oGkm_eF71HvhwAv7hXNyoA?p=%22adsense+account+merge%22&y=Search&fr=&ei=UTF-8
http://www.ask.com/web?qsrc=2417&o=0&l=dir&q=%22adsense+account+merge%22
Strangeness. |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
You're right. I don't think it's got anything to do with the domain though. I was wondering whether a kind of index / archive for forum threads was required. Technically, all forum posts are linked to and crawlable (via the "More posts" link) but I think the problem partially comes from the content on these numbered pages changing so frequently. |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Hmm, I think what the forum needs is a calendar feature similar to the one already available for older blog posts... that would make a better archive structure than just paging. |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Agreed. Would you place forum threads under the date they were created or the date of the last comment? (I would opt for the date they were created to avoid issues with posts jumping from month to month.) |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
There's now a brand new archive for forum posts, integrated into the blog post archive using tabs! Please check it out at
http://blogoscoped.com/calendar/ |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
This is excellent! I assume it's being cached from the database right now? Or is it excluding any forum threads which have also been blogged about? (If that's the case, I think they still need to be included as we still want them to be crawled.) |
/pd ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Phillpp, thats sweet!!
how about a search tab too , its easier to remember keywords then titles ? maybe a keywords and a drop down to to indicate start_Date and end_date criteria ? |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
BTW, it might be worthwhile adding a "No forum posts" message prior to May 2004 so that it doesn't look broken. (Or just remove the "Forum" tab.)
If we're adding items to the "wish list" :-) how about previous / next month links so that it's easy to keep going forwards / backwards between either blog / forum posts in case you can't find what you're looking for? (That would make a search box less important as you can just hit Ctrl+F and then keep hitting F3 on each page.) |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
It's supposed to exclude threads which start as reply to a blog post... only showing threads which started in the forum. But now that you mention it, let me check if it correctly includes those threads which only later were connected to a blog post, might well be there's a bug which excludes those right now. |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
> BTW, it might be worthwhile adding a "No forum posts" > message prior to May 2004 so that it doesn't look broken. > (Or just remove the "Forum" tab.)
Ah yeah, will do. |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
> It's supposed to exclude threads which start as reply to a blog post...
I'd argue that all forum threads should be shown (but with the same icon that appears in the forum page when they're replies to a blog post) as sometimes they have different titles and people might remember the original discussion rather than the blog post. |
Rohit Srivastwa ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Oh this is good. Loved the speed at which philipp added the feature :)
Philipp, think again of distributing this forum/blog software man! Even a paid version will do for many people :D |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Yes, threads which start in the forum should not be excluded. I was only talking about excluding those threads which didn't originate in the forum, but started out as replies to a post – those will also always have the same title as the blog post, by the way. (BUT, I can now confirm there is a bug which even excludes those threads which originated in the forum and then were merged with a blog post... currently looking into fixing that.) |
Niraj Sanghvi ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Thanks for adding that so fast, I'm already loving the tabs on the calendar :)
It might be useful (if it doesn't hit the database too hard) to include the number of replies in parentheses for each forum post so you could see at a glance which threads were really active. |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Update: I've added a stats page to the archive (this makes use of the Google Charts API): http://blogoscoped.com/calendar/stats |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Stats are cool! (I think the link is a bit hard to stumble across though.) |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
I notice that the forum posts now have the reply icon next to them if they originated in the forum but were blogged about:
http://blogoscoped.com/calendar/2008-02/forum
I'm not sure that makes sense as the reply icon implies they were a reply, whereas they were really a predecessor to the blog post. Perhaps it's time for another icon? One which implies "blogged"? That would also be useful on the forum page too I think, so it's clear whether it started as a forum post or a blog post. |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Yeah you're right, that icon can be misleading for anything else but meaning "reply". Will have to think about if there's a better icon that doesn't introduce too many different icons to understand. Alternatively, there's also always the option to just use a word, like "blogged". (I did this with "locked" already to avoid too many icons catching attention, also because manual locks to threads are rare so an icon couldn't be as easily learned.) |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Yeah, that would work too I guess. Although it's more common for a thread to get posted. An icon like this (which really represents "posted to a notice board") might be an idea otherwise:
static.ak.facebook.com/images/icons/post.gif
I think it would make sense to still include those in the RSS feed too. Otherwise, a thread might be in the RSS one minute and it could have disappeared the next if it gets blogged. |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Hmm. Have to think about this. After all even threads which originate in the forum may later on receive replies from the blog post, so the reply icon isn't all wrong. Perhaps it could be a modified reply icon. |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Of course. I'd forgotten about that... |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
> It might be useful (if it doesn't hit the database > too hard) to include the number of replies in > parentheses for each forum post so you could > see at a glance which threads were really active.
Just added a "hot" icon for threads of a certain size... that should help for now... |
Niraj Sanghvi ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
>Just added a "hot" icon for threads of a certain size... > that should help for now...
Perfect! Whatever you set for the threshold on that seems to be working really well. |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Tony elsewhere says... > More stats please!
So here goes – the stats page now has a search box where you can input any keyword to get a word frequency since 2003. http://blogoscoped.com/calendar/stats
(The search is very slow & resource hungry...) |
Tony Ruscoe ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
*e.g. gmail, cool, tony ruscoe...
Heh. I think caching the graphs like you do is a good idea as most people will enter the same things – e.g. google, gmail, gdrive... |
Philipp Lenssen ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
Bugfix: Had to adjust the grid lines, they were not in-sync with the years before. |
/pd ![[PersonRank 10] [PersonRank 10]](image/postrank/10.gif) | 17 years ago # |
dang that freakin stats is just to awsome . now I can see where all my time is being spent :(-
|