{"id":22205,"date":"2020-06-05T14:29:39","date_gmt":"2020-06-05T18:29:39","guid":{"rendered":"http:\/\/blumenthals.com\/blog\/?p=22205"},"modified":"2020-06-15T15:38:56","modified_gmt":"2020-06-15T19:38:56","slug":"the-worlds-best-bialys-and-local-searchs-worst-nightmare-robots-txt","status":"publish","type":"post","link":"https:\/\/blumenthals.com\/blog\/2020\/06\/05\/the-worlds-best-bialys-and-local-searchs-worst-nightmare-robots-txt\/","title":{"rendered":"The World’s Best Bialys and Local Search’s Worst Nightmare – Robots.txt"},"content":{"rendered":"

Update<\/strong>\u00a06\/14<\/strong>: Google has now started to index the site. As of now there are two of about 20 pages that have been indexed<\/a>.<\/p>\n

\"\"<\/p>\n

Update 6\/12:<\/strong> Good news. Kossars have fixed their robots.txt<\/a> file and removed the disallow function that prevented Google from indexing them. Onward and upward.<\/p>\n

User-agent: *<\/p>\n

***************************************************<\/p>\n

\"\"I was thinking about New York City today. For a million personal and historic reasons, when I think about NYC my mind goes to bialys. Not just any bialy. But the world’s best bialys, the ones by Kossars<\/a>.1<\/sup><\/p>\n

A bialy2<\/sup> for the non-initiate is best described as a bagel with character although it really isn’t a bagel at all. And Kossars, well Kossars is Kossars. A NYC landmark and a remnant of the Lower East Side Jewish ghettos of the last century3<\/sup>.<\/p>\n

So any ways back to my story. I thought about NY, bialys and Kossars and rather than ordering from Goldbelly, I wanted to see if I could avoid the middle man. So I searched for Kossar’s bialy<\/a>\u00a0to see if they shipped direct.<\/p>\n

What did I find? AARGH!<\/p>\n

\"\"<\/p>\n

A typical small business website (a nice looking one though) disabled by a robot.txt<\/a> file idiocy.<\/p>\n

User-agent: *
\nDisallow: \/<\/p>\n

I did what any red blooded citizen of jewish, New York descent would do who is desperate for some noshing… I called them and tried to explain the problem. \u00a0I spoke with the front counter, the manager and finally someone that seemed like the owner. Despite being clueless, she gave me her email and promised to share it with her webmaster.<\/p>\n

I sent along detailed instructions how to avoid the ignominy of the world’s easiest way to drop out of sight in local search. Let’s see if they get it fixed.<\/p>\n

1 \u00a0These bialys are so good that sometimes Andrew Shotland by way of thanks sends them my way for a large client I send his way. He is not cheap, he just appreciates the finer things in life.\u00a0<\/sup><\/p>\n

2. A bialy really is unique. No sweet, no raisins, a savory only tour de force with a very chewy crust, onions in a lump in the middle and a lot of holes that delivers butter AND\/OR cream chees & \u00a0lox better than any thing I have ever found. They are frequently made with lye.<\/a>\u00a0<\/sup><\/p>\n

3. According to Wikipedia<\/a> Kossar’s Bialys, originally known as Mirsky and Kossar’s<\/sup>[4]<\/a> <\/sup>when Isadore Mirsky and Morris Kossar founded it in 1936. I am afraid to ask what happened to Mirsky.<\/sup><\/p>\n","protected":false},"excerpt":{"rendered":"

Update\u00a06\/14: Google has now started to index the site. As of now there are two of about 20 pages that have been indexed. Update 6\/12: Good news. Kossars have fixed their robots.txt file and removed the disallow function that prevented Google from indexing them. Onward and upward. User-agent: * *************************************************** I was thinking about New …<\/p>\n","protected":false},"author":262,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24],"tags":[],"class_list":["post-22205","post","type-post","status-publish","format-standard","hentry","category-google-plus"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/posts\/22205","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/users\/262"}],"replies":[{"embeddable":true,"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/comments?post=22205"}],"version-history":[{"count":18,"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/posts\/22205\/revisions"}],"predecessor-version":[{"id":22232,"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/posts\/22205\/revisions\/22232"}],"wp:attachment":[{"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/media?parent=22205"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/categories?post=22205"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blumenthals.com\/blog\/wp-json\/wp\/v2\/tags?post=22205"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}