{"id":328,"date":"2019-03-06T12:50:34","date_gmt":"2019-03-06T11:50:34","guid":{"rendered":"http:\/\/justmakeit.es\/?p=328"},"modified":"2019-03-06T16:06:39","modified_gmt":"2019-03-06T15:06:39","slug":"incluir-una-nueva-columna-en-una-tabla-preexistente-en-impala-con-spark","status":"publish","type":"post","link":"http:\/\/justmakeit.es\/?p=328","title":{"rendered":"Incluir una nueva columna en una tabla preexistente en Impala con Spark"},"content":{"rendered":"\n<p>La principal consideraci\u00f3n a tener en cuenta para a\u00f1adir una nueva columna en una tabla que ya existe es que el m\u00e9todo no genera la columna sobre el DataFrame actual sino que la genera sobre uno nuevo.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\/\/ para poder utilizar la funci\u00f3n current_timestamp es necesario realizar la \n\/\/ importaci\u00f3n\nimport org.apache.spark.sql.functions._\n\n\/\/ se define la configuraci\u00f3n Spark\nval sparkConf = new SparkConf().setAppName(APP_NAME)\n\n\/\/ se define el StreamingContext con una frecuencia de 10 segundos    \nval ssc = new StreamingContext(sparkConf, Seconds(10))\n\n\/\/ se configura el HiveContext\nval hc = new HiveContext(ssc.sparkContext)\n\nval table:DataFrame = hc.table( tableName )\nval operacionDf:DataFrame = hc.read.schema( table.schema ).json( jsonRDD )\n\n\/\/ se a\u00f1ade el current_timestamp en una nueva columna creada para tal efecto\nval newDf:DataFrame = operacionDf.withColumn( \"aggr_time\", date_format( current_timestamp(), \"yyyy-MM-dd hh:mm:ss\" ) )\n\nnewDf.write.mode( SaveMode.Append ).insertInto( tableName )<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>La principal consideraci\u00f3n a tener en cuenta para a\u00f1adir una nueva columna en una tabla que ya existe es que &hellip; <a href=\"http:\/\/justmakeit.es\/?p=328\" class=\"btn btn-readmore\">Read More <span class=\"screen-reader-text\"> \u00abIncluir una nueva columna en una tabla preexistente en Impala con Spark\u00bb<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[37,34],"class_list":["post-328","post","type-post","status-publish","format-standard","hentry","category-programacion","tag-hive","tag-spark"],"_links":{"self":[{"href":"http:\/\/justmakeit.es\/index.php?rest_route=\/wp\/v2\/posts\/328","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/justmakeit.es\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/justmakeit.es\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/justmakeit.es\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/justmakeit.es\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=328"}],"version-history":[{"count":1,"href":"http:\/\/justmakeit.es\/index.php?rest_route=\/wp\/v2\/posts\/328\/revisions"}],"predecessor-version":[{"id":329,"href":"http:\/\/justmakeit.es\/index.php?rest_route=\/wp\/v2\/posts\/328\/revisions\/329"}],"wp:attachment":[{"href":"http:\/\/justmakeit.es\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=328"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/justmakeit.es\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=328"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/justmakeit.es\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=328"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}