New top story on Hacker News: Robots.txt for the NYT has a specific exclusion for an 1996 news article

Robots.txt for the NYT has a specific exclusion for an 1996 news article
46 by mikeortman | 7 comments on Hacker News.


Comments

Post a Comment

Popular Posts

NSF names two new Innovation Engines in North Carolina, announces $30 Million in new investments to region

NSF names two new Innovation Engines in North Carolina, announces $30 Million in new investments to region Today, the U.S. National Science Foundation announced two of the first-ever NSF Regional Innovation Engines (NSF Engines) as a part of President Biden’s Investing in America agenda. The Piedmont Triad Regenerative Medicine Engine and the North… Published January 26, 2024 at 11:00AM Read more at nsf.gov
Read more