G\u00fcn\u00fcm\u00fcz\u00fcn dijital d\u00fcnyas\u0131nda, uygulamalar ve sistemler taraf\u0131ndan \u00fcretilen log verileri, operasyonel g\u00f6r\u00fcn\u00fcrl\u00fck, g\u00fcvenlik analizi, performans izleme ve i\u015f zekas\u0131 i\u00e7in paha bi\u00e7ilmez bir kaynakt\u0131r. Ancak bu devasa ve s\u00fcrekli b\u00fcy\u00fcyen veri y\u0131\u011f\u0131n\u0131n\u0131 etkin bir \u015fekilde toplamak, depolamak ve analiz etmek, \u00f6zellikle geleneksel veri ambar\u0131 \u00e7\u00f6z\u00fcmleriyle zorlay\u0131c\u0131 olabilir. Bu makalede, Apache Iceberg’in modern veri g\u00f6l\u00fc yeteneklerini, Amazon S3’\u00fcn \u00f6l\u00e7eklenebilir depolama g\u00fcc\u00fcn\u00fc ve Amazon Data Firehose’un ger\u00e7ek zamanl\u0131 veri al\u0131m\u0131n\u0131 bir araya getirerek nas\u0131l g\u00fc\u00e7l\u00fc ve maliyet etkin bir log analitik platformu olu\u015fturulabilece\u011fini detayl\u0131ca inceleyece\u011fiz. Bu mimari, log verilerinizden maksimum de\u011feri elde etmenizi sa\u011flarken, esneklik ve performanstan \u00f6d\u00fcn vermez.<\/p>\n
Apache Iceberg, b\u00fcy\u00fck veri k\u00fcmeleri \u00fczerinde g\u00fcvenilir ve y\u00fcksek performansl\u0131 tablolar olu\u015fturmak i\u00e7in tasarlanm\u0131\u015f a\u00e7\u0131k kaynakl\u0131 bir tablo format\u0131d\u0131r. Geleneksel veri g\u00f6l\u00fc yakla\u015f\u0131mlar\u0131n\u0131n (\u00f6rne\u011fin, Hive tablolar\u0131) kar\u015f\u0131la\u015ft\u0131\u011f\u0131 bir\u00e7ok zorlu\u011fu a\u015farak, veri g\u00f6llerini veri ambar\u0131 benzeri \u00f6zelliklerle donat\u0131r.<\/p>\n
Iceberg, veri g\u00f6l\u00fc tablolar\u0131na ACID (Atomicity, Consistency, Isolation, Durability) \u00f6zelliklerini getirir. Bu, log verilerini eklerken, g\u00fcncellerken veya silerken veri b\u00fct\u00fcnl\u00fc\u011f\u00fcn\u00fc garanti eder. \u00d6zellikle, Firehose gibi ara\u00e7larla gelen log ak\u0131\u015flar\u0131n\u0131 g\u00fcvenilir bir \u015fekilde i\u015flemek i\u00e7in \u00f6nemlidir. Birden fazla yaz\u0131c\u0131n\u0131n ayn\u0131 tabloya ayn\u0131 anda veri yazmas\u0131 durumunda bile tutarl\u0131l\u0131k sa\u011flan\u0131r.<\/p>\n
Log verileri zamanla de\u011fi\u015fen \u015femalara sahip olabilir; yeni alanlar eklenebilir, mevcut alanlar de\u011fi\u015ftirilebilir veya kald\u0131r\u0131labilir. Iceberg, \u015fema evrimini sorunsuz bir \u015fekilde y\u00f6netir. Kullan\u0131c\u0131lar\u0131n tablolar\u0131 yeniden yazmas\u0131na gerek kalmadan \u015fema de\u011fi\u015fiklikleri yapmas\u0131na olanak tan\u0131r, bu da log analizi platformlar\u0131n\u0131n bak\u0131m\u0131n\u0131 ve geli\u015fimini b\u00fcy\u00fck \u00f6l\u00e7\u00fcde basitle\u015ftirir.<\/p>\n
Iceberg, verileri sorgu performans\u0131n\u0131 art\u0131rmak i\u00e7in otomatik olarak b\u00f6l\u00fcmlere ay\u0131rabilir. En \u00f6nemlisi, bu b\u00f6l\u00fcmleme detaylar\u0131n\u0131 kullan\u0131c\u0131dan gizler. Sorgu motorlar\u0131, verilerin nas\u0131l b\u00f6l\u00fcmlendi\u011fini bilmek zorunda kalmadan en uygun b\u00f6l\u00fcm\u00fc otomatik olarak bulur ve sorgular. Bu, log verilerini tarih, kaynak IP veya log seviyesi gibi kriterlere g\u00f6re sorgularken performans\u0131 art\u0131r\u0131r.<\/p>\n
Iceberg’in zaman yolculu\u011fu \u00f6zelli\u011fi, bir tablonun belirli bir zamandaki veya belirli bir anl\u0131k g\u00f6r\u00fcnt\u00fcdeki durumunu sorgulaman\u0131za olanak tan\u0131r. Bu, hatal\u0131 veri y\u00fcklemelerini geri almak, ge\u00e7mi\u015fteki log durumlar\u0131n\u0131 analiz etmek veya veri de\u011fi\u015fikliklerinin zaman i\u00e7indeki etkilerini incelemek i\u00e7in inan\u0131lmaz derecede faydal\u0131d\u0131r.<\/p>\n
Amazon S3 (Simple Storage Service), bulutta \u00f6l\u00e7eklenebilir, dayan\u0131kl\u0131 ve maliyet etkin nesne depolama hizmetidir. Log analitik platformumuzun temel depolama katman\u0131n\u0131 olu\u015fturur.<\/p>\n
S3, neredeyse s\u0131n\u0131rs\u0131z depolama kapasitesi sunar ve petabaytlarca log verisini kolayca bar\u0131nd\u0131rabilir. Y\u00fczde 99.999999999 (11 dokuz) veri dayan\u0131kl\u0131l\u0131\u011f\u0131 ile verilerinizin g\u00fcvende oldu\u011fundan emin olabilirsiniz. Bu, kritik log verileri i\u00e7in vazge\u00e7ilmez bir \u00f6zelliktir.<\/p>\n
S3, kullan\u0131lan depolama miktar\u0131na g\u00f6re \u00f6deme modeli sunar ve \u00e7e\u015fitli depolama s\u0131n\u0131flar\u0131 (Standard, Intelligent-Tiering, Infrequent Access, Glacier) ile maliyetleri optimize etme imkan\u0131 sa\u011flar. Bu, b\u00fcy\u00fck hacimli log verilerini uzun s\u00fcre saklamak i\u00e7in olduk\u00e7a ekonomiktir.<\/p>\n
S3, AWS ekosistemindeki di\u011fer bir\u00e7ok hizmetle (Athena, EMR, Glue, Firehose) sorunsuz bir \u015fekilde entegre olur. Bu entegrasyon, Iceberg tablolar\u0131n\u0131n S3 \u00fczerinde olu\u015fturulmas\u0131n\u0131 ve \u00e7e\u015fitli analiz ara\u00e7lar\u0131yla sorgulanmas\u0131n\u0131 kolayla\u015ft\u0131r\u0131r.<\/p>\n
Amazon Kinesis Data Firehose, ger\u00e7ek zamanl\u0131 ak\u0131\u015f verilerini veri g\u00f6llerine, veri ambarlar\u0131na ve analiz hizmetlerine g\u00fcvenilir bir \u015fekilde teslim etmek i\u00e7in tasarlanm\u0131\u015f tam olarak y\u00f6netilen bir hizmettir. Log analitik platformumuzda, log verilerini kaynaklardan toplay\u0131p S3’teki Iceberg tablolar\u0131na aktarmak i\u00e7in merkezi bir rol oynar.<\/p>\n
Firehose, veri ak\u0131\u015f\u0131n\u0131 y\u00f6netme, \u00f6l\u00e7eklendirme veya altyap\u0131 sa\u011flama ihtiyac\u0131n\u0131 ortadan kald\u0131r\u0131r. Log verilerini HTTP u\u00e7 noktalar\u0131, Kinesis Data Streams veya do\u011frudan AWS SDK’lar\u0131 arac\u0131l\u0131\u011f\u0131yla alabilir. Ard\u0131ndan, verileri tamponlayarak ve s\u0131k\u0131\u015ft\u0131rarak S3’e toplu olarak (batch) teslim eder.<\/p>\n
Firehose, S3’e veri teslim etmeden \u00f6nce AWS Lambda fonksiyonlar\u0131 kullanarak veri d\u00f6n\u00fc\u015ft\u00fcrme yetene\u011fi sunar. Bu, ham log verilerini ayr\u0131\u015ft\u0131rmak, zenginle\u015ftirmek veya belirli bir formata (\u00f6rne\u011fin, JSON’dan Parquet’e) d\u00f6n\u00fc\u015ft\u00fcrmek i\u00e7in kullan\u0131labilir. Iceberg tablolar\u0131 genellikle Parquet veya ORC format\u0131ndaki dosyalar\u0131 kulland\u0131\u011f\u0131ndan, Firehose’un bu d\u00f6n\u00fc\u015ft\u00fcrme yetene\u011fi olduk\u00e7a de\u011ferlidir.<\/p>\n
{\n \"DeliveryStreamName\": \"MyLogDeliveryStream\",\n \"DeliveryStreamType\": \"DirectPut\",\n \"S3DestinationConfiguration\": {\n \"BucketARN\": \"arn:aws:s3:::my-log-bucket\",\n \"Prefix\": \"raw-logs\/\",\n \"ErrorOutputPrefix\": \"error-logs\/\",\n \"BufferingHints\": {\n \"SizeInMBs\": 128,\n \"IntervalInSeconds\": 300\n },\n \"CompressionFormat\": \"GZIP\",\n \"EncryptionConfiguration\": {\n \"NoEncryptionConfig\": \"NoEncryption\"\n },\n \"CloudWatchLoggingOptions\": {\n \"Enabled\": true,\n \"LogGroupName\": \"\/aws\/kinesisfirehose\/MyLogDeliveryStream\",\n \"LogStreamName\": \"S3Delivery\"\n },\n \"ProcessingConfiguration\": {\n \"Enabled\": true,\n \"Processors\": [\n {\n \"Type\": \"Lambda\",\n \"Parameters\": [\n {\n \"ParameterName\": \"LambdaFunctionARN\",\n \"ParameterValue\": \"arn:aws:lambda:us-east-1:123456789012:function:LogProcessorFunction\"\n },\n {\n \"ParameterName\": \"BufferSizeInMBs\",\n \"ParameterValue\": \"3\"\n },\n {\n \"ParameterName\": \"BufferIntervalInSeconds\",\n \"ParameterValue\": \"60\"\n }\n ]\n }\n ]\n },\n \"FormatConversionConfiguration\": {\n \"Enabled\": true,\n \"InputFormatConfiguration\": {\n \"Deserializer\": {\n \"OpenXJsonSerDe\": {}\n }\n },\n \"OutputFormatConfiguration\": {\n \"Serializer\": {\n \"ParquetSerDe\": {\n \"Compression\": \"SNAPPY\"\n }\n }\n }\n }\n }\n}<\/pre>\n<\/code>
\nBu \u00f6rnek, Firehose'un loglar\u0131 al\u0131p bir Lambda fonksiyonuyla i\u015fledikten sonra Parquet format\u0131nda S3'e kaydetti\u011fini g\u00f6stermektedir.<\/p>\n
Iceberg Log Analitik Platformunun Mimarisi<\/h3>\n
Bu platformun temel mimarisi a\u015fa\u011f\u0131daki bile\u015fenlerden olu\u015fur:<\/p>\n
1. Log Kaynaklar\u0131:<\/strong> Uygulamalar, sunucular, a\u011f cihazlar\u0131, konteynerler (\u00f6rne\u011fin, EC2, ECS, EKS).
\n2. Veri Toplama:<\/strong> Loglar\u0131 Firehose'a g\u00f6ndermek i\u00e7in kullan\u0131lan ara\u00e7lar (Fluentd, Fluent Bit, CloudWatch Agent, SDK'lar).
\n3. Amazon Kinesis Data Firehose:<\/strong> Ger\u00e7ek zamanl\u0131 log ak\u0131\u015flar\u0131n\u0131 al\u0131r, iste\u011fe ba\u011fl\u0131 olarak d\u00f6n\u00fc\u015ft\u00fcr\u00fcr ve S3'e teslim eder.
\n4. Amazon S3:<\/strong> Ham ve i\u015flenmi\u015f log verileri i\u00e7in dayan\u0131kl\u0131 ve \u00f6l\u00e7eklenebilir depolama katman\u0131. Iceberg tablolar\u0131n\u0131n temelini olu\u015fturur.
\n5. AWS Glue Data Catalog:<\/strong> Iceberg tablolar\u0131n\u0131n meta verilerini (\u015fema, konum vb.) y\u00f6netir. Spark, Athena gibi sorgu motorlar\u0131 bu katalogdan faydalan\u0131r.
\n6. Apache Iceberg Tablolar\u0131:<\/strong> S3 \u00fczerindeki log verilerini yap\u0131land\u0131r\u0131lm\u0131\u015f, sorgulanabilir formatta sunar.
\n7. Sorgu ve Analiz Motorlar\u0131:<\/strong>
\n * Amazon Athena:<\/strong> S3 \u00fczerindeki Iceberg tablolar\u0131n\u0131 standart SQL kullanarak sunucusuz bir \u015fekilde sorgulamak i\u00e7in idealdir.
\n * Apache Spark (AWS EMR veya Glue):<\/strong> B\u00fcy\u00fck \u00f6l\u00e7ekli veri i\u015fleme, d\u00f6n\u00fc\u015f\u00fcm ve daha karma\u015f\u0131k analizler i\u00e7in kullan\u0131l\u0131r.
\n * Trino (PrestoSQL):<\/strong> \u00c7e\u015fitli veri kaynaklar\u0131 \u00fczerinde h\u0131zl\u0131, interaktif sorgular i\u00e7in kullan\u0131labilir.
\n8. G\u00f6rselle\u015ftirme ve Raporlama:<\/strong> Amazon QuickSight, Grafana veya di\u011fer BI ara\u00e7lar\u0131, analiz sonu\u00e7lar\u0131n\u0131 g\u00f6rselle\u015ftirmek i\u00e7in.<\/p>\nVeri Ak\u0131\u015f\u0131:<\/strong>
\nLoglar, kaynaklardan Firehose'a g\u00f6nderilir. Firehose, loglar\u0131 toplar, Lambda ile d\u00f6n\u00fc\u015ft\u00fcr\u00fcr (iste\u011fe ba\u011fl\u0131 olarak JSON'dan Parquet'e \u00e7evirir) ve s\u0131k\u0131\u015ft\u0131r\u0131lm\u0131\u015f Parquet dosyalar\u0131 olarak S3'e kaydeder. Bu Parquet dosyalar\u0131, Iceberg tablosunun veri dosyalar\u0131 haline gelir. AWS Glue Data Catalog, Iceberg tablosunun meta verilerini y\u00f6netir. Son olarak, Athena veya Spark gibi sorgu motorlar\u0131, Glue Catalog \u00fczerinden Iceberg tablosuna eri\u015ferek log verilerini sorgular ve analiz eder.<\/p>\nUygulama Ad\u0131mlar\u0131 ve Konfig\u00fcrasyon \u00d6rnekleri<\/h3>\n
Platformu kurmak i\u00e7in temel ad\u0131mlar \u015funlard\u0131r:<\/p>\n
1. Amazon S3 Kovas\u0131 Olu\u015fturma<\/h4>\n
Log verileriniz ve Iceberg meta verileri i\u00e7in bir S3 kovas\u0131 olu\u015fturun.<\/p>\n
aws s3 mb s3:\/\/my-iceberg-log-bucket --region us-east-1<\/pre>\n<\/code><\/p>\n
2. Amazon Data Firehose Teslim Ak\u0131\u015f\u0131 Olu\u015fturma<\/h4>\n
Loglar\u0131 S3'e aktaracak bir Firehose teslim ak\u0131\u015f\u0131 olu\u015fturun. D\u00f6n\u00fc\u015f\u00fcm i\u00e7in bir Lambda fonksiyonu kullanabilirsiniz.<\/p>\n
* Kaynak:<\/strong> Direct Put veya Kinesis Data Stream
\n* Hedef:<\/strong> Amazon S3
\n* S3 Kovas\u0131:<\/strong> my-iceberg-log-bucket<\/code>
\n* \u00d6n Ek:<\/strong> raw-logs\/<\/code> (ham loglar i\u00e7in) veya processed-logs\/<\/code> (i\u015flenmi\u015f loglar i\u00e7in)
\n* Veri D\u00f6n\u00fc\u015ft\u00fcrme:<\/strong> Lambda fonksiyonu kullanarak loglar\u0131 ayr\u0131\u015ft\u0131r\u0131n ve yap\u0131land\u0131r\u0131lm\u0131\u015f bir formata (\u00f6rn. JSON) d\u00f6n\u00fc\u015ft\u00fcr\u00fcn.
\n* Format D\u00f6n\u00fc\u015ft\u00fcrme:<\/strong> Parquet format\u0131na d\u00f6n\u00fc\u015ft\u00fcrmeyi etkinle\u015ftirin.<\/p>\n3. AWS Glue Data Catalog'da Iceberg Tablosu Olu\u015fturma<\/h4>\n
Log verilerini sorgulamak i\u00e7in bir Iceberg tablosu tan\u0131mlay\u0131n. AWS Glue Catalog, Iceberg tablolar\u0131n\u0131n meta verilerini depolamak i\u00e7in kullan\u0131labilir. Bu tabloyu genellikle Apache Spark veya Amazon Athena'da olu\u015ftururuz.<\/p>\n
Spark ile Iceberg Tablosu Olu\u015fturma \u00d6rne\u011fi:<\/strong><\/p>\nfrom pyspark.sql import SparkSession\n\nspark = SparkSession.builder \\\n .appName(\"IcebergLogTable\") \\\n .config(\"spark.sql.catalog.glue\", \"org.apache.iceberg.spark.SparkSessionCatalog\") \\\n .config(\"spark.sql.catalog.glue.warehouse\", \"s3:\/\/my-iceberg-log-bucket\/warehouse\") \\\n .config(\"spark.sql.catalog.glue.catalog-impl\", \"org.apache.iceberg.aws.glue.GlueCatalog\") \\\n .config(\"spark.sql.catalog.glue.io-impl\", \"org.apache.iceberg.aws.s3.S3FileIO\") \\\n .getOrCreate()\n\n# \u00d6rnek bir \u015fema tan\u0131mlay\u0131n (log verilerinizin yap\u0131s\u0131na g\u00f6re)\nspark.sql(\"\"\"\nCREATE TABLE glue.logs.web_access_logs (\n timestamp TIMESTAMP,\n level STRING,\n message STRING,\n ip_address STRING,\n user_agent STRING,\n request_method STRING,\n request_uri STRING,\n status_code INT\n)\nUSING iceberg\nPARTITIONED BY (days(timestamp), level)\nLOCATION 's3:\/\/my-iceberg-log-bucket\/web_access_logs';\n\"\"\")\n\n# Firehose'dan gelen verileri tabloya yazmak i\u00e7in (\u00f6rne\u011fin, bir Glue Job ile)\n# spark.read.parquet(\"s3:\/\/my-iceberg-log-bucket\/processed-logs\/\").writeTo(\"glue.logs.web_access_logs\").append()<\/pre>\n<\/code>
\nBu \u00f6rnekte, glue.logs.web_access_logs<\/code> ad\u0131nda bir Iceberg tablosu olu\u015fturulur ve timestamp<\/code> ile level<\/code> alanlar\u0131na g\u00f6re b\u00f6l\u00fcmlenir.<\/p>\n4. Log Verilerini Firehose'a G\u00f6nderme<\/h4>\n
Uygulamalar\u0131n\u0131zdan veya sunucular\u0131n\u0131zdan loglar\u0131 Firehose'a g\u00f6nderin. \u00d6rne\u011fin, Python SDK kullanarak:<\/p>\n
import boto3\nimport json\n\nfirehose_client = boto3.client('firehose', region_name='us-east-1')\n\nlog_entry = {\n \"timestamp\": \"2023-10-27T10:00:00Z\",\n \"level\": \"INFO\",\n \"message\": \"User 'testuser' logged in successfully.\",\n \"ip_address\": \"192.168.1.100\",\n \"user_agent\": \"Mozilla\/5.0\",\n \"request_method\": \"GET\",\n \"request_uri\": \"\/login\",\n \"status_code\": 200\n}\n\nresponse = firehose_client.put_record(\n DeliveryStreamName='MyLogDeliveryStream',\n Record={\n 'Data': json.dumps(log_entry) + '\\n'\n }\n)\nprint(response)<\/pre>\n<\/code><\/p>\n
5. Log Verilerini Sorgulama ve Analiz Etme (Athena ile)<\/h4>\n
Iceberg tablosu olu\u015fturulduktan ve Firehose ile veri ak\u0131\u015f\u0131 sa\u011fland\u0131ktan sonra, Amazon Athena'y\u0131 kullanarak Iceberg tablolar\u0131n\u0131 sorgulayabilirsiniz.<\/p>\n
-- Son 24 saatteki hata loglar\u0131n\u0131 sorgulama\nSELECT timestamp, message, ip_address\nFROM glue.logs.web_access_logs\nWHERE level = 'ERROR'\n AND timestamp >= current_timestamp - INTERVAL '1' DAY\nORDER BY timestamp DESC;\n\n-- En \u00e7ok hata \u00fcreten IP adreslerini bulma\nSELECT ip_address, COUNT(*) as error_count\nFROM glue.logs.web_access_logs\nWHERE level = 'ERROR'\nGROUP BY ip_address\nORDER BY error_count DESC\nLIMIT 10;<\/pre>\n<\/code><\/p>\n
Avantajlar ve Kullan\u0131m Senaryolar\u0131<\/h3>\n
Bu mimarinin sundu\u011fu temel avantajlar ve kullan\u0131m senaryolar\u0131 \u015funlard\u0131r:<\/p>\n
Avantajlar:<\/h4>\n
* \u00d6l\u00e7eklenebilirlik:<\/strong> Petabaytlarca log verisini depolayabilir ve i\u015fleyebilir.
\n* Maliyet Etkinli\u011fi:<\/strong> S3'\u00fcn d\u00fc\u015f\u00fck depolama maliyetleri ve sunucusuz servislerin (Firehose, Athena) kulland\u0131k\u00e7a \u00f6de modeli sayesinde maliyetleri optimize eder.
\n* Performans:<\/strong> Iceberg'in gizli b\u00f6l\u00fcmleme ve dosya organizasyonu sayesinde h\u0131zl\u0131 sorgu performans\u0131.
\n* Veri Tutarl\u0131l\u0131\u011f\u0131:<\/strong> ACID garantileri ile g\u00fcvenilir log analizi.
\n* Esneklik:<\/strong> \u015eema evrimi ve zaman yolculu\u011fu \u00f6zellikleri sayesinde de\u011fi\u015fen ihtiya\u00e7lara kolayca uyum sa\u011flar.
\n* Ger\u00e7ek Zamanl\u0131ya Yak\u0131n Analiz:<\/strong> Firehose ile loglar neredeyse ger\u00e7ek zamanl\u0131 olarak analize haz\u0131r hale gelir.<\/p>\nKullan\u0131m Senaryolar\u0131:<\/h4>\n
* Operasyonel \u0130zleme:<\/strong> Uygulama hatalar\u0131n\u0131, performans sorunlar\u0131n\u0131 ve sistem olaylar\u0131n\u0131 ger\u00e7ek zamanl\u0131ya yak\u0131n olarak izleme.
\n* G\u00fcvenlik Analizi:<\/strong> Yetkisiz eri\u015fim denemeleri, \u015f\u00fcpheli etkinlikler ve g\u00fcvenlik ihlallerini tespit etme.
\n* Denetim ve Uyumluluk:<\/strong> Yasal ve d\u00fczenleyici gereklilikler i\u00e7in log kay\u0131tlar\u0131n\u0131 uzun s\u00fcre saklama ve kolayca sorgulama.
\n* \u0130\u015f Zekas\u0131:<\/strong> Kullan\u0131c\u0131 davran\u0131\u015flar\u0131n\u0131 anlama, \u00fcr\u00fcn kullan\u0131m\u0131 analizi ve pazarlama kampanyalar\u0131n\u0131n etkinli\u011fini \u00f6l\u00e7me.
\n* Hata Ay\u0131klama ve Sorun Giderme:<\/strong> \u00dcretim ortam\u0131ndaki sorunlar\u0131n k\u00f6k nedenini h\u0131zl\u0131ca bulma.<\/p>\nSonu\u00e7: Gelece\u011fin Log Analiti\u011fi \u00c7\u00f6z\u00fcm\u00fc<\/h3>\n
Apache Iceberg, Amazon S3 ve Amazon Data Firehose kombinasyonu, modern log analiti\u011fi platformlar\u0131 i\u00e7in g\u00fc\u00e7l\u00fc, esnek ve maliyet etkin bir \u00e7\u00f6z\u00fcm sunar. Bu mimari, log verilerinin b\u00fcy\u00fck \u00f6l\u00e7ekte toplanmas\u0131n\u0131, depolanmas\u0131n\u0131 ve analiz edilmesini basitle\u015ftirirken, veri tutarl\u0131l\u0131\u011f\u0131, \u015fema evrimi ve zaman yolculu\u011fu gibi geli\u015fmi\u015f \u00f6zelliklerle veri g\u00f6llerinin potansiyelini maksimize eder. \u0130ster operasyonel g\u00f6r\u00fcn\u00fcrl\u00fck, ister g\u00fcvenlik izleme veya i\u015f zekas\u0131 ama\u00e7l\u0131 olsun, bu platform, log verilerinizden de\u011ferli i\u00e7g\u00f6r\u00fcler elde etmenizi sa\u011flayarak kurulu\u015funuzun veri odakl\u0131 karar alma s\u00fcre\u00e7lerini g\u00fc\u00e7lendirecektir.<\/p>\n
S\u0131k\u00e7a Sorulan Sorular (SSS)<\/h3>\n1. Apache Iceberg, geleneksel Hive tablolar\u0131na g\u00f6re ne gibi avantajlar sunar?<\/h4>\n
Iceberg, Hive tablolar\u0131n\u0131n \u00f6tesinde ACID i\u015flemleri, \u015fema evrimi, gizli b\u00f6l\u00fcmleme, zaman yolculu\u011fu ve daha g\u00fcvenilir veri garantileri sunar. Bu \u00f6zellikler, \u00f6zellikle s\u00fcrekli de\u011fi\u015fen ve b\u00fcy\u00fcyen log verileri i\u00e7in daha sa\u011flam ve y\u00f6netilebilir bir \u00e7\u00f6z\u00fcm sa\u011flar.<\/p>\n
2. Amazon Data Firehose yerine Amazon Kinesis Data Streams kullanabilir miyim?<\/h4>\n
Evet, kullanabilirsiniz. Kinesis Data Streams, daha d\u00fc\u015f\u00fck gecikme s\u00fcresi ve daha fazla kontrol gerektiren senaryolar i\u00e7in uygundur. Ancak, Kinesis Data Streams'i y\u00f6netmek Firehose'a g\u00f6re daha fazla operasyonel y\u00fck getirir. Firehose, genellikle log analiti\u011fi gibi senaryolarda \"tamamen y\u00f6netilen\" yap\u0131s\u0131 nedeniyle tercih edilir.<\/p>\n
3. Bu platformu kurmak i\u00e7in hangi AWS hizmetlerine ihtiyac\u0131m var?<\/h4>\n
Temel olarak Amazon S3, Amazon Data Firehose ve AWS Glue Data Catalog'a ihtiyac\u0131n\u0131z olacakt\u0131r. Sorgulama i\u00e7in Amazon Athena veya Apache Spark (AWS EMR\/Glue \u00fczerinde) kullanabilirsiniz. Veri d\u00f6n\u00fc\u015ft\u00fcrme i\u00e7in AWS Lambda da faydal\u0131 olabilir.<\/p>\n
4. Iceberg tablolar\u0131ndaki verileri nas\u0131l g\u00fcncelleyebilir veya silebiliriz?<\/h4>\n
Iceberg, veri g\u00f6l\u00fc tablolar\u0131nda UPDATE<\/code>, DELETE<\/code> ve MERGE INTO<\/code> gibi SQL i\u015flemlerini destekler. Bu i\u015flemler genellikle Spark gibi i\u015flem motorlar\u0131 arac\u0131l\u0131\u011f\u0131yla ger\u00e7ekle\u015ftirilir. Bu, GDPR gibi veri y\u00f6netimi gereksinimlerini kar\u015f\u0131lamak i\u00e7in \u00f6nemlidir.<\/p>\n5. Bu mimari ne kadar maliyetli olabilir?<\/h4>\n
Maliyet, depolanan veri miktar\u0131na (S3), Firehose \u00fczerinden ge\u00e7en veri miktar\u0131na, Athena veya Spark ile yap\u0131lan sorgular\u0131n karma\u015f\u0131kl\u0131\u011f\u0131na ve s\u0131kl\u0131\u011f\u0131na ba\u011fl\u0131d\u0131r. Ancak, S3'\u00fcn d\u00fc\u015f\u00fck depolama maliyetleri ve sunucusuz hizmetlerin kulland\u0131k\u00e7a \u00f6de modeli sayesinde, geleneksel veri ambar\u0131 \u00e7\u00f6z\u00fcmlerine g\u00f6re genellikle \u00e7ok daha maliyet etkin bir \u00e7\u00f6z\u00fcm sunar.<\/p>\n","protected":false},"excerpt":{"rendered":"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma\nG\u00fcn\u00fcm\u00fcz\u00fcn dijital d\u00fcnyas\u0131nda, uygulamalar ve sistemler t…","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"csco_page_header_type":"","csco_page_load_nextpost":"","csco_page_subscribe_form":"","csco_page_contact_form":"","footnotes":""},"categories":[874],"tags":[],"class_list":{"0":"post-36788","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-server","7":"cs-entry","8":"cs-video-wrap"},"yoast_head":"\n
Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma - Kodlar\u0131n Gizemli D\u00fcnyas\u0131<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/\" \/>\n<meta property=\"og:locale\" content=\"tr_TR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma\" \/>\n<meta property=\"og:description\" content=\"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma G\u00fcn\u00fcm\u00fcz\u00fcn dijital d\u00fcnyas\u0131nda, uygulamalar ve sistemler t...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/\" \/>\n<meta property=\"og:site_name\" content=\"Kodlar\u0131n Gizemli D\u00fcnyas\u0131\" \/>\n<meta property=\"article:published_time\" content=\"2025-12-23T00:30:31+00:00\" \/>\n<meta name=\"author\" content=\"Fatih Soysal\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Yazan:\" \/>\n\t<meta name=\"twitter:data1\" content=\"Fatih Soysal\" \/>\n\t<meta name=\"twitter:label2\" content=\"Tahmini okuma s\u00fcresi\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 dakika\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/\"},\"author\":{\"name\":\"Fatih Soysal\",\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1\"},\"headline\":\"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma\",\"datePublished\":\"2025-12-23T00:30:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/\"},\"wordCount\":2093,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1\"},\"articleSection\":[\"Server\"],\"inLanguage\":\"tr\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#respond\"]}],\"copyrightYear\":\"2025\",\"copyrightHolder\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/\",\"url\":\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/\",\"name\":\"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma - Kodlar\u0131n Gizemli D\u00fcnyas\u0131\",\"isPartOf\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#website\"},\"datePublished\":\"2025-12-23T00:30:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#breadcrumb\"},\"inLanguage\":\"tr\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Anasayfa\",\"item\":\"https:\/\/fatihsoysal.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#website\",\"url\":\"https:\/\/fatihsoysal.com\/blog\/\",\"name\":\"Fatihsoysal.com\",\"description\":\"Blog - Yaz\u0131l\u0131m D\u00fcnyas\u0131 Tecr\u00fcbelerim\",\"publisher\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/fatihsoysal.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"tr\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1\",\"name\":\"Fatih Soysal\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"tr\",\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/fatihsoysal.com\/blog\/wp-content\/uploads\/2024\/04\/cropped-replicate-prediction-3kgg1hgjn5rgp0cf0p5tr0jw7w-1.png\",\"contentUrl\":\"https:\/\/fatihsoysal.com\/blog\/wp-content\/uploads\/2024\/04\/cropped-replicate-prediction-3kgg1hgjn5rgp0cf0p5tr0jw7w-1.png\",\"width\":512,\"height\":512,\"caption\":\"Fatih Soysal\"},\"logo\":{\"@id\":\"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/image\/\"},\"description\":\"Kullan\u0131m ve kodlama m\u00fckemmeliyetini odak alan uygulamalar olu\u015fturma deneyimine sahip, profesyonel olarak 15+ y\u0131l \u00fczeri deneyime sahip bir yaz\u0131l\u0131m m\u00fchendisi.\",\"url\":\"https:\/\/fatihsoysal.com\/blog\/author\/fatihsoysal\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma - Kodlar\u0131n Gizemli D\u00fcnyas\u0131","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/","og_locale":"tr_TR","og_type":"article","og_title":"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma","og_description":"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma G\u00fcn\u00fcm\u00fcz\u00fcn dijital d\u00fcnyas\u0131nda, uygulamalar ve sistemler t...","og_url":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/","og_site_name":"Kodlar\u0131n Gizemli D\u00fcnyas\u0131","article_published_time":"2025-12-23T00:30:31+00:00","author":"Fatih Soysal","twitter_card":"summary_large_image","twitter_misc":{"Yazan:":"Fatih Soysal","Tahmini okuma s\u00fcresi":"12 dakika"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#article","isPartOf":{"@id":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/"},"author":{"name":"Fatih Soysal","@id":"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1"},"headline":"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma","datePublished":"2025-12-23T00:30:31+00:00","mainEntityOfPage":{"@id":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/"},"wordCount":2093,"commentCount":0,"publisher":{"@id":"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1"},"articleSection":["Server"],"inLanguage":"tr","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#respond"]}],"copyrightYear":"2025","copyrightHolder":{"@id":"https:\/\/fatihsoysal.com\/blog\/#organization"}},{"@type":"WebPage","@id":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/","url":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/","name":"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma - Kodlar\u0131n Gizemli D\u00fcnyas\u0131","isPartOf":{"@id":"https:\/\/fatihsoysal.com\/blog\/#website"},"datePublished":"2025-12-23T00:30:31+00:00","breadcrumb":{"@id":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#breadcrumb"},"inLanguage":"tr","potentialAction":[{"@type":"ReadAction","target":["https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/fatihsoysal.com\/blog\/apache-iceberg-s3-ve-amazon-data-firehose-ile-guclu-bir-log-analitik-platformu-olusturma\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Anasayfa","item":"https:\/\/fatihsoysal.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Apache Iceberg, S3 ve Amazon Data Firehose ile G\u00fc\u00e7l\u00fc Bir Log Analitik Platformu Olu\u015fturma"}]},{"@type":"WebSite","@id":"https:\/\/fatihsoysal.com\/blog\/#website","url":"https:\/\/fatihsoysal.com\/blog\/","name":"Fatihsoysal.com","description":"Blog - Yaz\u0131l\u0131m D\u00fcnyas\u0131 Tecr\u00fcbelerim","publisher":{"@id":"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/fatihsoysal.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"tr"},{"@type":["Person","Organization"],"@id":"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/002a254750921dcfd568a99e48240dd1","name":"Fatih Soysal","image":{"@type":"ImageObject","inLanguage":"tr","@id":"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/fatihsoysal.com\/blog\/wp-content\/uploads\/2024\/04\/cropped-replicate-prediction-3kgg1hgjn5rgp0cf0p5tr0jw7w-1.png","contentUrl":"https:\/\/fatihsoysal.com\/blog\/wp-content\/uploads\/2024\/04\/cropped-replicate-prediction-3kgg1hgjn5rgp0cf0p5tr0jw7w-1.png","width":512,"height":512,"caption":"Fatih Soysal"},"logo":{"@id":"https:\/\/fatihsoysal.com\/blog\/#\/schema\/person\/image\/"},"description":"Kullan\u0131m ve kodlama m\u00fckemmeliyetini odak alan uygulamalar olu\u015fturma deneyimine sahip, profesyonel olarak 15+ y\u0131l \u00fczeri deneyime sahip bir yaz\u0131l\u0131m m\u00fchendisi.","url":"https:\/\/fatihsoysal.com\/blog\/author\/fatihsoysal\/"}]}},"yoast_meta":{"yoast_wpseo_title":"","yoast_wpseo_metadesc":"","yoast_wpseo_canonical":""},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/posts\/36788","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/comments?post=36788"}],"version-history":[{"count":0,"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/posts\/36788\/revisions"}],"wp:attachment":[{"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/media?parent=36788"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/categories?post=36788"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fatihsoysal.com\/blog\/wp-json\/wp\/v2\/tags?post=36788"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}