Deploying a Modern Security Data Lake Solve Legacy SIEM Problems, Integrate Data Science, and Enable Collaboration David Baum REPORT Compliments ofDavid BaumDeploying a Modern Security Data Lake Solve Legacy SIEM Problems, Integrate Data Science, and Enable Collaboration Boston Farnham Sebastopol Tokyo Beijing Boston Farnham Sebastopol Tokyo Beijing978-1-098-13495-2 [LSI]Deploying a Modern Security Data Lake by David Baum Copyright © 2022 O’Reilly Media Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles ( http://oreilly.com ). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com . Acquisitions Editor: Nicole Butterfield Development Editor: Gary O’Brien Production Editor: Kate Galloway Copyeditor: nSight, Inc.Proofreader: Jonathon Owen Interior Designer: David Futato Cover Designer: Randy Comer Illustrator: Kate Dullea July 2022: First Edition Revision History for the First Edition 2022-07-21: First Release The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Deploying a Modern Security Data Lake , the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and Snowflake. See our state‐ ment of editorial independence .Table of Contents Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v 1.The Rise of the Security Data Lake. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Understanding the Limitations of the Traditional SIEM Model 2 Expanding Y our Analytic Horizons 4 Reviewing Security Data Lake Prototypes 5 Introducing the Modern Cloud Security Data Lake 6 Harnessing the Power of a Cloud Data Platform and Connected Ecosystem 7 Summary 8 2.Implementing a Security Data Lake. . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Phase 1: Assess Y our Current State 11 Phase 2: Collect and Migrate Data 13 Phase 3: Establish and Verify Analytics 14 Roles and Responsibilities 16 Summary 17 3.Connecting Best-of-Breed Security Applications. . . . . . . . . . . . . . . . 19 Understanding the Connected Applications Model 21 Context Matters 22 Counting the C