We announced Canary Watch a year ago as a coalition project to list Warrant Canaries and monitor them for changes or removal. Canary Watch was a joint project, with EFF, Freedom of the Press Foundation, NYU Law, Calyx and the Berkman Center.
Along the way, the project has been part of the massive popularization of the concept: we began with just eleven canaries listed, and now just a year later we have almost seventy. In the course of tracking those, we have learned many lessons about the different types of canaries that are present on the web, as well as what happens when a canary goes away.
In that way, the Canary Watch project has been a major success, and we’ve decided that it has achieved the goals we set out for it. As of today we will no longer accept submissions of new canaries or monitor the existing canaries for changes or take downs.
Transparency reports and warrant canaries have an important role to play in the fight against illegal and unconstitutional national security process, including National Security Letters and other secret court processes. We have not received any court orders or government requests to shut down the Canary Watch project. Rather, all of the members of the Canary Watch coalition have come to the agreement that the project has run its course and has come to a natural ending point.
Over the course of the project, we have learned some things about the nature of canaries on the Web which are important for anyone working with warrant canaries or doing activism around them.The Number of Warrant Canaries Is Increasing
We started with eleven canaries; within four months that number grew to fifty; at the end of the project there were almost seventy warrant canaries in the Canary Watch database, with requests to add dozens more. In the last month the number of searches for warrant canaries grew by an order of magnitude. This is likely thanks in large part to the disappearance of reddit's warrant canary from their 2016 transparency report. The last year has, without a doubt, been a banner year for awareness of warrant canaries.
Search frequency for the term “Warrant canary” since 2007 Warrant Canaries Provide Interesting, But Not Definitive Information
Since July of 2013 Pinterest has been publishing a warrant canary which simply read "National Security: 0" as a part of their quarterly transparency report. In 2015, Pinterest's number of national security requests changed from 0 to 0-249, reported for January to June, and July to December (instead of quarterly). What prompted this move? Under the law, a company that has received a national security request can report in bands of 250, starting at 0, semiannually. Thus, there is certainly the strong implication that Pinterest did receive a national security request, because it would have otherwise have continued to report 0.
Yet, in our time working with Canary Watch we have seen many canaries go away and come back, fail to be updated, or disappear altogether along with the website that was hosting it. Until the gag orders accompanying national security requests are struck down as unconstitutional, there is no way to know for certain whether a canary change is a true indicator. Instead the reader is forced to rely on speculation and circumstantial evidence to decide what the meaning of a missing or changed canary is.Warrant Canaries Can Be Fickle
We also observed warrant canaries behaving in unexpected ways. Sometimes a canary would have subtle changes in language or grammar, which can be hard to interpret. Other canaries would regularly change what URL they were located at, and for others domains these URL changes were sudden and unexpected. Canaries often were not updated at all, or were updated several days or weeks late. Sometimes the warrant canary, along with the entire website would disappear without explanation or reason, and sometimes just the warrant canary would disappear and come back later, unchanged. All of this uncertainty caused numerous false alarms, which made it difficult to monitor warrant canaries. Additionally, this chaos served as a further demonstration of how difficult it is to interpret what it means when a warrant canary changes.Warrant Canaries Come In Many Shapes and Sizes
One of the most surprising things that we have learned over the course of the Canary Watch project is that almost every canary is unique. We have seen canaries that were in PDFs, plaintext, HTML, and even images. We have seen canaries that were integrated into the website banner and canaries which were only available on Github. We have seen canaries that are signed using GPG, canaries that are part of a transparency report, canaries that include the day's weather and top news headlines. We have seen canaries that are updated on a daily basis and canaries which are updated once per year. We have seen canaries that were created once and then never updated again. Again, the fact that canaries are non-standard makes it difficult to automatically monitor them for changes or takedowns.
The major strides in our understanding about the nature and current status of warrant canaries and national security letters mean Canary Watch has definitely been a success. Moreover, it raised awareness and contributed to an important policy debate that is now well underway. In contrast to the uncertainty a year ago, it now seems that the Internet at large can offer robust and decentralized monitoring of warrant canaries; the rapid spread of the news when reddit’s canary disappeared is a testament to that fact.
Finally we would like to give a huge thank you to our coalition partners on this project for the last year: The Calyx Institute, Freedom of the Press Foundation, The Berkman Center, and the NYU School of Law.
Update May 26, 2016: Senate Judiciary Committee Chair Charles Grassley (R-IA) postponed marking up the Email Privacy Act. The committee website will provide further details on whether or not the bill will be marked up in June.
The Senate Judiciary Committee is expected to vote on the Email Privacy Act on Thursday. Senators Patrick Leahy (D-VT) and Mike Lee (R-UT) plan to introduce near-identical text of the House-passed bill, H.R. 699, as substitute language for the existing Senate bill, S. 356. This manager’s amendment contains minor changes. In addition, up to eight different amendments may be offered.
The Email Privacy Act would amend the Electronic Communications Privacy Act (ECPA) to require the government to get a probable cause warrant from a judge before obtaining private content stored in the “cloud” with companies such as Google, Facebook, and Dropbox. The House of Representatives passed H.R. 699 last month by a unanimous vote of 419-0. The Senate Judiciary Committee held a hearing last September on the need to reform ECPA and codify the Sixth Circuit Court of Appeals’ 2010 ruling that the government violated the Fourth Amendment when it obtained emails stored by third parties without a probable cause warrant.
EFF recommends senators vote "NO" on all amendments except the manager's amendment, and HEN16524 by Sen. Jeff Flake (R-AZ), which would loosen the gag requirements of ECPA that are contested in a recent lawsuit by Microsoft.
The committee must reject the other amendments, especially one by Senators Lindsey Graham (R-SC), Sheldon Whitehouse (D-RI), and Richard Blumenthal (D-CT) that would "reform" the Computer Fraud and Abuse Act (OLL16603); and the electronic communication transactional records (ECTR) amendment (OLL16601) by Sen. John Cornyn (R-TX) that would expand the types of information the FBI can obtain with a National Security Letter, without prior judicial oversight.
The committee must also defeat amendments that would create a so-called mandatory emergency exception—a requirement that service providers comply with government requests for user data when the government claims emergency circumstances, again without prior oversight by a court. The mandatory emergency exception is found in two amendments by Sen. Jeff Sessions (R-AL). EFF recommends senators vote "NO" on both amendments: HEN16527 and HEN16529. These amendments are unnecessary because ECPA (18 U.S.C. § 2702) already permits service providers to hand over content or other records in an “emergency involving danger of death or serious physical injury to any person.” And it is beneficial for users that service providers may withhold data when they believe that the government is fraudulently using the emergency exception to bypass due process requirements. In 2010, for example, the Department of Justice’s Inspector General found that “exigent letters and informal requests were used in circumstances that do not appear to qualify as emergencies under Section 2702” (p. 261).
We urge the Senate Judiciary Committee to pass a “clean” bill without any further amendments that would weaken the privacy protections in the legislation. Please contact your senators and urge them to pass a strong Email Privacy Act so that your emails and private documents stored online have the same protection as those stored in your home or office!
At this month's Sydney Drupal meet up I did a presentation about Search in Drupal 8. In the video, I explain three ways you can create a search page, they are as follows.
1. Core Search
The core Search module which comes with Drupal has some new functionality in Drupal 8. The biggest change is the ability to create custom search pages without using any other module.
2. Views Filter
A common way to build search pages in Drupal 7 was to create a views page and use the "Search Keywords" filter in views. This can still be done in Drupal 8 and best of all Views is now part of core.
3. Search API
The Search API module is used to create powerful search pages and it's highly extensible. It is the module to learn and use for building search pages.
Over the last 19 years, the Open Source Initiative (OSI) has been the steward of the Open Source Definition (or OSD), establishing a common language when discussing what it means to be an Open Source license, and a list of licenses which are known to be compatible with the OSD.
This is taken to its logic next step this year, with the OSI providing a machine readable publication of OSI approved licenses at api.opensource.org. This will allow third parties to become license-aware, and give organizations the ability to clearly determine if a license is, in fact, an Open Source license, from the authoritative source regarding Open Source licenses, the OSI.
Open Source Lead at GitHub, Brandon Keepers offered, "A canonical, machine-readable source of license metadata is a great step towards enabling developers to build tools around open source licensing and compliance. We can't wait to see what the community does with it."
The concept behind this API is to be a "hub" to store a central list of crosswalks and common identifiers to other services, allowing third parties who are already license-aware to provide their mappings, and pull OSI approval status programatically. As a proof of concept, SPDX identifiers have been added, trivially allowing cross-walks to SPDX datasets. This allows anyone to take an SPDX license ID, and determine if it's OSI approved by asking the OSI API.
If you have ideas on additional metadata to add to the License specification, please feel free to file a bug (or send a patch!) with the licenses repo, with some snippets of data as an example.
I am glad to announce that we will have a blockchain specialist from Society for Worldwide Interbank Financial Telecommunication (also known as SWIFT) for the regular weekly meeting.
He will explain the society’s approach for the blockchain technology. He will also explain how to utilize ISO20022 for the entire blockchain community.
The link for SWIFT meeting is as following:
NOTE: The link is different from the usual link we have been using for the past few weeks.
As you may know, Drupal 6 has reached End-of-Life (EOL) which means the Drupal Security Team is no longer doing Security Advisories or working on security patches for Drupal 6 core or contrib modules - but the Drupal 6 LTS vendors are and we're one of them!
Today, there is a Moderately Critical security release for XML sitemap to fix a Cross-Site Scripting (XSS) vulnerability.
The module doesn't sufficiently filter the URL when it is displayed in the sitemap.
This vulnerability is mitigated if the setting for "Include a stylesheet in the sitemaps for humans." on the module's administration settings page is not enabled (the default is enabled).
If you have a Drupal 6 site using the XML sitemap, we recommend you update immediately! We have already deployed the patch for all of our Drupal 6 Long-Term Support clients. :-)
If you'd like all your Drupal 6 modules to receive security updates and have the fixes deployed the same day they're released, please check out our D6LTS plans.
Note: if you use the myDropWizard module (totally free!), you'll be alerted to these and any future security updates, and will be able to use drush to install them (even though they won't necessarily have a release on Drupal.org).
It's my pleasure to introduce Acquia Bolt, a development tool for generating new Drupal projects using a template derived from our Professional Services' best practices.
We began building and using Bolt internally over the past year. Our goal was to codify a set of tools and conventions that would allow us to:Tags: acquia drupal planet
This episode we hit a land mark of twenty episodes, so instead of picking on Mario the entire time, we talk about our favorite moments at Drupalcon in New Orleans. Of course this is funny since half of us didn't actually attend the 'Con. Other episode titles considered: "Bob and Mario live vicariously through Mark and Ryan"
Other than a quick sample of blocking a specific file extension using the 'nomask' option, the documenttation for Drupal's file_scan_directory() does not help much with how to bock content from certain directories. The documentation says
So it leaves you to believe that any regex should work. So setting
$options['nomask'] = "#(/deleted/)#"
should block any directory named 'deleted'. The problem is, it doesn't work that way. In file_scan_directory() the regex is not run against the full path of the file, it is only run against the directory or filename recursively. It is not evaluating 'directory1/subsection/deleted/index.html' , where the regex above would definitely come back with a hit and reject the item. It is first evaluating, 'directory', then 'subsection', then 'deleted'... It does not get a hit on deleted because it is missing the slashes on each side.
One possibilty would be to remove the slashes from the regex like this:
$options['nomask'] = "#(deleted)#";
But without the slashes, it would not only reject the directory 'deleted', it would reject the directory 'not-deleted' and the file 'faq-why-my-account-was-deleted.htm' which might have undesired consequences.
The trick to get it to reject only 'deleted' as a directory and its contents is to restrict the regex to only the start and end of the string being evaluated. like this
$options['nomask'] = "#^(deleted)$#";
and if you wanted to block 'deleted' and another directory like '_vti_cnf' it would look like this:
$options['nomask'] = "#^(deleted|_vti_cnf)$#";
While I have had the privilege of attending a number of DrupalCons and camps over the years, I cannot remember one with as many sessions and BOFs (birds of a feather) on the topic of security. In addition to the security talk on the program schedule, I had a great time chatting with individuals in the hallways and a few security focused companies in the exhibit hall.
Once upon a time, I had a coworker named Bob who, when he needed help, would start the conversation in the middle and work to both ends. My phone would ring, and the first thing I heard was: “Hey, so, we need the spreadsheets on Tuesday so that Information Security can have them back to us in time for the estimates.”
Spreadsheets? Estimates? Bob and I had never discussed either. As I had been “discouraged” from responding with “What the hell are you talking about now?” I spent the next 10 minutes of every Bob call trying to tease out the context of his proclamations.
Clearly, Bob needed help—and not just with spreadsheets.
Then there was Susan. When Susan wanted help, she gave me the entire life story of a project in the most polite, professional language possible. An email from Susan might go like this:
I’m working on the Super Bananas project, which we started three weeks ago and have been slowly working on since. We began with persona writing, then did some scenarios, and discussed a survey.
[Insert two more paragraphs of the history of the project]
I’m hoping—if you have the opportunity (due to your previous experience with [insert four of my last projects in chronological order])—you may be able to share a content-inventory template that would be appropriate for this project. If it isn’t too much trouble, when you get a chance, could you forward me the template at your earliest convenience?
Thank you in advance for your cooperation,
An email that said, “Hey do you have a content-inventory template I could use on the Super Bananas Project?” would have sufficed, but Susan wanted to be professional. She believed that if I had to ask a question, she had failed to communicate properly. And, of course, that failure would weigh heavy on all our heads.
Bob and Susan were as opposite as the tortoise and the hare, but they shared a common problem. Neither could get over the river and through the woods effectively. Specifically, they were both lousy at establishing context and getting to the point.
We all need the help of others to build effective tools and applications. Communication skills are so critical to that endeavor that we’ve seen article after article after article—not to mention books, training classes, and job postings—stressing the importance of communication skills. Without the ability to communicate, we can neither build things right, nor build the right things, for our clients and our users.
Still, context-setting is a tricky skill to learn. Stray too far toward Bob, and no one knows what we’re talking about. Follow Susan’s example, and people get bored and wander off before we get to the point.
Whether we’re asking a colleague for help or nudging an end user to take action, we want them to respond a certain way. And whether we’re writing a radio ad, publishing a blog post, writing an email, or calling a colleague, we have to set the proper level of context to get the result we want.
The most effective technique I’ve found for beginners is a process I call “Once Upon a Time.”Fairy tales? Seriously?
Fairy tales are one of our oldest forms of folklore, with evidence indicating that they may stretch back to the Roman Empire. The prelude “Once upon a time” dates to 1380 BCE, according to the Oxford English Dictionary. Wikipedia lists over 75 language variations of the stock story opener. It’s safe to say that the vast majority of us, regardless of language or culture, have heard our share of fairy tales, from the 1800s-era Brothers Grimm stories to the 1987 musical Into the Woods.
We know how they go:
Once upon a time, there was a [main character] living in [this situation] who [had this problem]. [Some person] knows of this need and sends the [main character] out to [complete these steps]. They [do things] but it’s really hard because [insert challenges]. They overcome [list of challenges], and everyone lives happily ever after.
Fairy tales are effective oral storytelling techniques precisely because they follow a standard structure that always provides enough context to understand the story. Almost everything we do can be described with this structure.
Once upon a time Anne lacked an ice cream sandwich. This forced her to get off the couch and go to the freezer, where food stayed amazingly cold. She was forced to put her hands in the icy freezer to dig the ice cream sandwich box out of the back. She overcame the cold and was rewarded with a tasty ice cream sandwich! And they all lived happily ever after.
The structure of a fairy tale’s beginning has a lot of similarities to the journalistic Five Ws of basic information gathering: Who? What? When? Where? Why? How?
In our communication construct, we are the main character whose situation and problem need to be succinctly described. We’ve been sent out to do a thing, we’ve hit a challenge, and now we need specific help to overcome the challenge.How does this help me if I’m a Bob or a Susan?
When Bob wanted to tell his story, he didn’t start with “Once upon a time…” He started halfway through the story. If Bob was Little Red Riding Hood, he would have started by saying, “We need scissors and some rocks.” (Side note: the general lack of knowledge about how surgery works in that particular tale gives me chills.)
When Susan wanted to tell her story, she started before “Once upon a time…” If she was Little Red Riding Hood, she started by telling you how her parents met, how long they dated, and so on, before finally getting around to mentioning that she was trapped in a wolf’s stomach.
When we tell our stories, we have to start at the beginning—not too early, not too late. If we’re Bob, that means making sure we’ve relayed the basic facts: who we are, what our goal is, possibly who sent us, and what our challenge is. If we’re Susan, we need to make sure we limit ourselves to the facts we actually need.
This is where we take the fairy-tale format and put it into the first person. Susan might write:
Once upon a time, the Bananas team asked me to do the content strategy for their project. We made good progress until we had this problem: we don’t have a template for content inventories. Bob suggested I contact you. Do you have a template you can send us?
Bob might say:
Once upon a time, you and I were working on the data mapping of the new Information Security application. Then Information Security asked us to send the mapping to them so they could validate it. This is a problem because we only have until Tuesday to give them the unfinished spreadsheets. Otherwise we’ll hit an even bigger problem: we won’t be able to estimate the project size on Friday without the spreadsheet. Can you help me get the spreadsheet to them on time?
Notice the parallels between the fairy tales and these drafts: we know the main character, their situation, who sent them or triggered their move, and what they need to solve their problem. In Bob’s case, this is much more information than he usually provides. In Susan’s, it’s probably much less. In both cases, we’ve distilled the situation and the request down to the basics. In both cases, the only edit needed is to remove “Once upon a time…” from the first sentence, and it’s ready to go.But what about…?
Both the Bobs and the Susans I’ve worked with have had questions about this technique, especially since in both cases they thought they were already doing a pretty good job of providing context.
The original Susan had two big concerns that led her to giving out too much information. The first was that she’d sound unprofessional if she didn’t include every last detail and nuance of business etiquette. The second was that if her recipient had questions, they’d consider her amateurish for not providing every bit of information up front.
Susans of the world, let me assure you: clear, concise communication is professional. The message isn’t not to use “please” and “thank you”; it’s that “If it isn’t too much trouble, when you get a chance, could you please consider…” is probably overkill.
Beyond that, no one can anticipate every question another person might have. Clear communication starts a dialogue by covering the basics and inviting questions. It also saves time; you only have to answer the questions your colleague or reader actually have. If you’re not sure whether to keep a piece of information in your story, take it out and see if the tale still makes sense.
Bob was a tougher nut to crack, in part because he frequently didn’t realize he was starting in the middle. Bob was genuinely baffled that colleagues hadn’t read his mind to know what he was talking about. He thought he just needed the answer to one “quick” question. Once he was made aware that he was confusing—and sometimes annoying—coworkers, he could be brought back on track with gentle suggestions. “Okay Bob, let’s start over. Once upon a time you were…?”Begin at the beginning and stop at the end
Using the age-old format of “Once upon a time…” gives us an incredibly sturdy framework to use for requesting action from people. We provide all of the context they need to understand our request, as well as a clear and concise description of that request.
Clear, concise, contextual communication is professional, efficient, and much less frustrating to everyone involved, so it pays to build good habits, even if the basis of those habits seems a bit corny.
Do you really need to start with “Once upon a time…” to tell a story or communicate a request? Well, it doesn’t hurt. The phrase is really a marker that you’re changing the way you think about your writing, for whom you’re writing it, and what you expect to gain. Soup doesn’t require stones, and business communication doesn’t require “Once upon a time…”
But it does lead to more satisfying endings.
And they all lived happily ever after.