Pros and Cons to ACLs in the Search Index

I recently provided an answer to an online discussion group (LinkedIn Enterprise Search Engine Professionals Forum) and thought it might be helpful for some of our customers, so I wanted to share it directly through the Attivio blog.

The question posted by Bob Lawrence:

I noticed the GSA announcement talked about ACLs in the index to allow "early binding" to security rules. This was also alluded to in connection to another discussion on open source security. In that thread, it was implied that for new security rules to take affect for SOLR/LUCENE, the search index needed to be rebuilt (please correct me if I misinterpreted the discussion). Is this also true of the GSA and other search engines that tout early binding to security rules? How big a deal is this? I am assuming organizations need to update the index on at least a daily basis to account for added/deleted documents. Can they do this without rebuilding the index and thus not incorporating new security rules?

The original question asked makes a good point about ingestion issues when ACLs are bound directly to documents. It isn't great when you have to re-ingest a large PDF just because the permissions on it changed. But there are even bigger issues when taking into account user and group changes.

For example, suppose the "Company Group" is made up of the "Consulting Group" and the "Sales Group". If the "Consulting Group" is removed from the "Company Group", you don't want to have to re-ingest all documents that have access to the "Company Group" just because the group hierarchy changed.

Then there's also the issue of users and ACLs existing on different realms. For example, if Tom@Windows and Tom@Unix are both the same user, it would be best if both realms' ACLs are applied when Tom queries.

All of these issues point out why it's best to separately store:

• Users, groups and their membership
• Documents
• Document ACLs

Warning: shameless plug follows ...

As was pointed out by another participant in the discussion, Kevin Watters, Attivio has had built-in support for JOINs as well as ACLs for quite some time, which has allowed us to secure queries using early binding without re-indexing documents. We have recently released a new security module with advanced user and group support, which makes this process even easier and handles all the issues mentioned. The security module comes with multiple features, including:

• An Active Directory scanner to ingest users, groups and group membership
• A Windows file scanner to ingest documents along with their Windows ACLs
• A Windows service to persist ACL updates in real-time to the index (without re-indexing the document)
• An Exchange scanner to ingest emails along with their ACLs
• A SharePoint scanner to ingest SharePoint documents along their ACLs
• The ability to create cross-realm aliases, so that user Tom@Windows and Tom@Unix can be treated as the same user with respect to permissions
• An extensible API to support custom user, group and ACL ingestion
• Complete security filtering of queries and their results (including facets) so users have no knowledge of applied security rules

I'd like to hear your thoughts on this topic as well. I think the hybrid approach using JOIN has changed this paradigm significantly.

If you’d like more information on the Attivio security model, check out our on-demand webinar called “Information Access Control: Can you really have faster, safer AND cheaper?”.

Author Bio

Since graduating from Carnegie Mellon University with a Master's degree in Information Networking, Greg George has had extensive experience developing high performance, large-scale data processing solutions. Greg was an early employee at Ab Initio Software Corporation where he solely designed and developed their database interface engine for all of the major database vendors. After Ab Initio, Greg worked at Lumigent Technologies as a technical leader designing auditing solutions for Oracle, DB2 and Sybase. Greg is a Principal Software Engineer working at Attivio.

TrackBack URI for this entry

Comments (0)