Classic Search Change - substrings no longer work

IanO · 3 Apr 2014, 01:57 PM

It took me a while to figure out why I couldn't quickly find some emails by typing part of a word. From the blog (April 3rd 2014):

"However the general consensus from classic users is that they’d really like the improved search that the current interface comes with. So today we’ve rolled out a change that better unifies the search syntax on both the classic and current interfaces."

What consensus? Guess I am not part of that.

If you type part of a word the you know is in some emails it will not find them. Now you need to type the whole word. It will find a substring if you type substr:exam (if you were looking for example). But that defeats the point.

Chipper · 14 Apr 2014, 02:58 AM

I've been experiencing problems with searching as well.

I used to type "faceb" and it would return all email notifications from Facebook. This is no longer working. Does anyone know a way to get this to work again? I'm using Classic Interface.

pjwalsh · 14 Apr 2014, 06:33 AM

That is correct, substring search is no longer the default in the Classic search. The blog post explains how unadorned text is handled:

It will do a fast indexed search of the from, to, cc, bcc and subject headers as well as the message body content... This search is done on words/terms with stemming where possible, not sub-strings. This searches the current folder by default on classic, and across all folders by default on the current interface. On classic, you can check the “All” checkbox to search across all folders.

If you want to revert to the historical sub-string searching of headers, you can use the substr: modifier. Some more examples:

example – fuzzy search from/to/cc/bcc/subject headers and message bodies for the word “example”

body:example – fuzzy search message bodies for the word “example”

to:example – fuzzy search to/cc/bcc headers for the word “example”

onlycc:example – fuzzy search cc header for the word “example”

substr:example – search from/to/cc/subject headers (but not body content) for the substring “example”

substr:(dinner john) – search from/to/cc/subject headers for both substrings “dinner” and “john”

substr:(body:example) – search message body content for the substring “example” (warning: likely very slow!)

substr:(to:example) - search to/cc/bcc headers for the substring “example”

substr:(onlycc:example) - search cc header for the substring “example”

To the blog post list should have been added:

from:example

Which is the simple behaviour (scope) I came to expect in the old Classic search.

And from the Help page on mailbox searching:

Phrases are matched on a complete basis with stemming (all different forms of the same word are treated as equivalent). This means, for example, a search for bus will match "buses" but not "business".
...
Using substr to wrap other search terms changes their behaviour. Instead of matching stemmed whole words, search terms will be matched on a straightforward "contains" basis. This means that even partial words or parts within words will match. For example, substr:(from:ber) will match an email from "Robert", because "ber" is contained in "Robert". Please note that searching for substrings will be significantly slower than regular searches.

--
Initially I was annoyed by the change, as looking for emails from a sender was my most common need, but I quickly came to appreciate the speed and value of a quick search of both headers and body.

Stem searches are much faster than searching by substring.

The keyboard shortcut for search is / (slash).

--
http://en.wikipedia.org/wiki/Stemming

pjwalsh · 14 Apr 2014, 09:16 AM

Quote:

Originally Posted by Chipper

I used to type "faceb" and it would return all email notifications from Facebook. This is no longer working. Does anyone know a way to get this to work again? I'm using Classic Interface.

from:facebook

IanO · 18 Apr 2014, 02:55 AM

@Chipper

This was my original point; I used to be able to type a simple short string, but now I need to add a modifier or type a whole word. Instead of just typing faceb, now you need to type (see pjwalsh's posts):

from:facebook

Just typing facebook should also work.

You should also be able to use:

substr:faceb

to get the original behavior.

@pjwalsh

Thanks for the great explanation. It's still annoying; guess I will just have to learn to type better

pjwalsh · 18 Apr 2014, 10:30 PM

Actually, it takes just two keystrokes

/ , f

to present a search box dropdown of all prior entries starting with f

from:facebook

will be first if it was your last used search,

select, hit enter

done.

Quick!

IanO · 22 Apr 2014, 09:21 AM

Hmm, thanks for the tip, not working for me though. Probably all the browser saved strings for the search box (I have that feature switched off).

NumberSix · 4 Jun 2014, 05:24 PM

I had been frustrated by the apparent lobotomization of the search function on Classic one too many times, and came here looking for answers...

Just wanna say thanks, pjwalsh, for the extremely helpful explanations!

As usual, things have actually gotten better, while I've failed to notice...

hankfoner · 19 Jun 2014, 04:25 AM

I had occasion to search for a certain address today on the new interface. I couldn't find the mail though I knew it was there. Thus I came upon this thread and the whole issue of stemming and substrings. I can see some of the advantages of stemming but believe that most people searching for an email address in the INBOX would prefer the substring search. However I think it is too much to ask the ordinary user to remember "substr: (....)". Why cannot there be a radio button in the search box to enable a choice of search methods?
Hank

NJSS · 1 Jul 2014, 09:56 PM

May I reinforce Hank's suggestion above - surely it would not be difficult to implement?

NJSS

wrycoder · 12 Nov 2014, 05:28 AM

The new search by "whole word" is really weird. It seems to find words with a shorter root than what I search for. e.g. I'm searching for PyconES and it finds all the mail with Pycon<whatever>. Searching for Pycon gives the same list, appending 'e' gives same list.

"subject

ycones" gives the same long list - everything with a word Pycon\w+

That was a colon p, btw!

I don't think search should stem on words shorter than the term submitted.

substr does work.

regex search would be nice.

3 Apr 2014, 01:57 PM	#1
IanO Senior Member Join Date: Oct 2003 Posts: 102	Classic Search Change - substrings no longer work It took me a while to figure out why I couldn't quickly find some emails by typing part of a word. From the blog (April 3rd 2014): "However the general consensus from classic users is that they’d really like the improved search that the current interface comes with. So today we’ve rolled out a change that better unifies the search syntax on both the classic and current interfaces." What consensus? Guess I am not part of that. If you type part of a word the you know is in some emails it will not find them. Now you need to type the whole word. It will find a substring if you type substr:exam (if you were looking for example). But that defeats the point.

14 Apr 2014, 06:33 AM	#3
pjwalsh Essential Contributor Join Date: Dec 2008 Location: Canada Posts: 312	That is correct, substring search is no longer the default in the Classic search. The blog post explains how unadorned text is handled: It will do a fast indexed search of the from, to, cc, bcc and subject headers as well as the message body content... This search is done on words/terms with stemming where possible, not sub-strings. This searches the current folder by default on classic, and across all folders by default on the current interface. On classic, you can check the “All” checkbox to search across all folders. If you want to revert to the historical sub-string searching of headers, you can use the substr: modifier. Some more examples: example – fuzzy search from/to/cc/bcc/subject headers and message bodies for the word “example” body:example – fuzzy search message bodies for the word “example” to:example – fuzzy search to/cc/bcc headers for the word “example” onlycc:example – fuzzy search cc header for the word “example” substr:example – search from/to/cc/subject headers (but not body content) for the substring “example” substr:(dinner john) – search from/to/cc/subject headers for both substrings “dinner” and “john” substr:(body:example) – search message body content for the substring “example” (warning: likely very slow!) substr:(to:example) - search to/cc/bcc headers for the substring “example” substr:(onlycc:example) - search cc header for the substring “example” To the blog post list should have been added: from:example Which is the simple behaviour (scope) I came to expect in the old Classic search. And from the Help page on mailbox searching: Phrases are matched on a complete basis with stemming (all different forms of the same word are treated as equivalent). This means, for example, a search for bus will match "buses" but not "business". ... Using substr to wrap other search terms changes their behaviour. Instead of matching stemmed whole words, search terms will be matched on a straightforward "contains" basis. This means that even partial words or parts within words will match. For example, substr:(from:ber) will match an email from "Robert", because "ber" is contained in "Robert". Please note that searching for substrings will be significantly slower than regular searches. -- Initially I was annoyed by the change, as looking for emails from a sender was my most common need, but I quickly came to appreciate the speed and value of a quick search of both headers and body. Stem searches are much faster than searching by substring. The keyboard shortcut for search is / (slash). -- http://en.wikipedia.org/wiki/Stemming Last edited by pjwalsh : 14 Apr 2014 at 09:58 AM.

18 Apr 2014, 02:55 AM	#5
IanO Senior Member Join Date: Oct 2003 Posts: 102	@Chipper This was my original point; I used to be able to type a simple short string, but now I need to add a modifier or type a whole word. Instead of just typing faceb, now you need to type (see pjwalsh's posts): from:facebook Just typing facebook should also work. You should also be able to use: substr:faceb to get the original behavior. @pjwalsh Thanks for the great explanation. It's still annoying; guess I will just have to learn to type better

18 Apr 2014, 10:30 PM	#6
pjwalsh Essential Contributor Join Date: Dec 2008 Location: Canada Posts: 312	Actually, it takes just two keystrokes / , f to present a search box dropdown of all prior entries starting with f from:facebook will be first if it was your last used search, select, hit enter done. Quick!

19 Jun 2014, 04:25 AM	#9
hankfoner Essential Contributor Join Date: Jul 2007 Location: Jerusalem, Israel Posts: 370	Searching I had occasion to search for a certain address today on the new interface. I couldn't find the mail though I knew it was there. Thus I came upon this thread and the whole issue of stemming and substrings. I can see some of the advantages of stemming but believe that most people searching for an email address in the INBOX would prefer the substring search. However I think it is too much to ask the ordinary user to remember "substr: (....)". Why cannot there be a radio button in the search box to enable a choice of search methods? Hank

14 Apr 2014, 02:58 AM	#2
Chipper Master of the @ Join Date: Oct 2003 Location: Greenbelt, MD (USA) Posts: 1,278	I've been experiencing problems with searching as well. I used to type "faceb" and it would return all email notifications from Facebook. This is no longer working. Does anyone know a way to get this to work again? I'm using Classic Interface.

22 Apr 2014, 09:21 AM	#7
IanO Senior Member Join Date: Oct 2003 Posts: 102	Hmm, thanks for the tip, not working for me though. Probably all the browser saved strings for the search box (I have that feature switched off).

4 Jun 2014, 05:24 PM	#8
NumberSix Cornerstone of the Community Join Date: Jan 2003 Location: The Village Posts: 605	I had been frustrated by the apparent lobotomization of the search function on Classic one too many times, and came here looking for answers... Just wanna say thanks, pjwalsh, for the extremely helpful explanations! As usual, things have actually gotten better, while I've failed to notice...

1 Jul 2014, 09:56 PM	#10
NJSS Master of the @ Join Date: Jul 2002 Location: Hampshire, UK Posts: 1,238	May I reinforce Hank's suggestion above - surely it would not be difficult to implement? NJSS

12 Nov 2014, 05:28 AM	#11
wrycoder Junior Member Join Date: Jan 2004 Posts: 29	The new search by "whole word" is really weird. It seems to find words with a shorter root than what I search for. e.g. I'm searching for PyconES and it finds all the mail with Pycon<whatever>. Searching for Pycon gives the same list, appending 'e' gives same list. "subjectycones" gives the same long list - everything with a word Pycon\w+ That was a colon p, btw! I don't think search should stem on words shorter than the term submitted. substr does work. regex search would be nice.