互联网大厂面试真题附含答案.docx
![资源得分’ title=](/images/score_1.gif)
![资源得分’ title=](/images/score_1.gif)
![资源得分’ title=](/images/score_1.gif)
![资源得分’ title=](/images/score_1.gif)
![资源得分’ title=](/images/score_05.gif)
《互联网大厂面试真题附含答案.docx》由会员分享,可在线阅读,更多相关《互联网大厂面试真题附含答案.docx(92页珍藏版)》请在得力文库 - 分享文档赚钱的网站上搜索。
1、互联网大厂面试真题附含答案-合并-2021目录大厂面试(1) .docx大厂面试(2) .docx大厂面试(3) .docx 面试真题目录.docx 高频面试题-.docx面试简述基于Flume数据采集流程数据源kafka | RabbitMQlOQ日志AgentHBaseSource Interceptor - Kafka Channel fHDFSHive二次开发flume各个组件(因为flume默认的一些功能还不能满足开发需求,如需要在 flume拦截器中处理中文乱码问题,更加灵活的Sink)启动命令为 nohup bin/flume-ng agent -n al -c conf -f
2、conf/flume-conf.properties -Dflume.monitoring.type=http -Dflume.monitoring.port=41414 &启动时监听41414端口,后台zabbix对该端口进行监控,有异常邮件告警curl localhost:41414/metrics | grep Channelsize 可以查看各个 Channel 之间的数据积压 情况Flume开发示例代码:1.使用官方的组件,搭配一个 从netcat source - filechannel - logger sink 的 demo# example.conf: A single-no
3、de Flume configuration# Name the components on this agent al.sources = rl al.sinks = klal.channelscl# Describe/configure the source al.sources.rl.type = netcat al.sources.rl.bind = localhostal.sources.rl.port = 44444# Describe the sink al.sinks.kl.type = logger# Use a channel which buffers events in
4、 memory al.channels.cl.type = memoryal.channels.cl.capacity = 1000al.channels.cl.transactionCapacity = 100# Bind the source and sink to the channel al.sources.rl.channels = cl al.sinks.kl.channel = cl输出结果:OK2018-11-13 14:14:56,786 (lifecycleSupervisor-1-2) INFO -org.apache.flume.source.NetcatSource.
5、start(NetcatSource.java:164)Createdserversocket:sun.nio.ch.ServerSocketChannelImpl/127.0.0.1:44444 2018-11-13 14:15:34,795 (SinkRunner-PollingRunner-DefaultSinkProcessor) INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:70) Event: headers:(body: 61 61 61 0Daaa. 2.编写自定义source ,抓取模拟数据经过
6、channel-logger sinkpackage flume.plugin;import java.nio.charset.Charset;public class Customsource extends Abstractsource implements Configurabl e,PollableSource0Overridepublic Status process() throws EventDeliveryException Random random = new Random();int randomNum = random.nextlnt(100);String text
7、= Hello world + random.nextlnt(100);HashMap header = new HashMap();header.put(id,Integer.toString(randomNum); 模拟数据this , getChannelProcessor() processEvent(EventBuilder.withBodyCtextCharset.forName(U TF-8),header);return Status.READY; )Override public void configure(Context context) ) 配置文件如下 al.sour
8、ces = rlal.sinks = kl al.channels = cl# # Describe/configure the sourceal.sources.rl.type = flume.plugin.Customsource # # Describe the sink al.sinks.kl.type = logger # # Use a channel which buffers events in memoryal.channels.cl.type = memory al.channels.cl.capacity = 1000 al.channels.cl.transaction
9、capacity = 100# Bind the source and sink to the channel al.sources.rl.channels = cl al.sinks.kl.channel = cl3.编写自定义的拦截器,过滤source接收的数据3.1 过滤包含XXX的数据public class Custominterceptor implements Interceptor Override public void initialize() / TODO Auto-generated method stub )Overridepublic Event intercept
10、(Event event) if(newString(event.getBody().contains(xxx) re turn null;);return event; )Overridepublic List intercept(List events) for(Event e:events) intercept(e);) return events; ) Override public void close() )Builder拦截器创建类public class CustomlnterceptorBuilder implements Builder(Override public vo
11、id configure(Context context) / TODO Auto-generated method stub)0Overridepublic Interceptor build() / TODO Auto-generated method stub return new Custominterceptor();)配置文件如下rlal.sources =al.sinks = sial.channels = clal.sources.rl.type = netcatal.sources.rl.bind = localhostal.sources.rl.port = 44444al
12、.sources.rl.interceptors = ilal, sources.rl.interceptors il. type =flume. plugin.CustomlnterceptorBuilder#flume.plugin.CustomInterceptor$CustomInterceptorBuilderal.sources.rl.interceptors.il.perserveExisting = trueal.sinks.si.type = loggeral.channels.cl.type = memoryal.channels.cl.capacity = 2al.cha
13、nnels.cl.transactionCapacity = 2al.sources.rl.channels = clal.sinks.si.channel = cl自定义headerpublic class CustomCountInterceptor implements Interceptor(private final String headerKey;private static final String CONF_HEADER_KEY = header1;private static final String DEFAULT_HEADER = count”;private fina
14、l AtomicLong currentcount;public CustomCountInterceptor(Context ctx) headerKey = ctx.getString(CONF_HEADER_KEY,DEFAULT_HEADER); currentcount = new AtomicLong();)Overridepublic void initialize() / TODO Auto-generated method stub )Overridepublic Event intercept(Event event) long count = currentcount.i
15、ncrementAndGet(); event.getHeaders(),put(headerKey, String.valueOf(count); return event;)Overridepublic List intercept(List events) for(Event e:events) intercept(e);return events;Overridepublic void close() )public static class CustomlnterceptorBuilder implements Builder private Context ctx;Override
16、public Interceptor build() return new CustomCountlnterceptor(ctx);)Overridepublic void configure(Context context) this.ctx = context;)4.编写自定义sink ,将数据接入mysql数据库中public class CustomSink extends Abstractsink implements Configurableprivate Connection connect;private Statement stmt;private String column
17、Name;private String url;private String user;private String password;private String tableName;Overridepublic synchronized void start() try Class . forName(com mysql.jdbc.Driver); connect = DriverManager.getConnection(url user,password); stmt = connect.c reateStatement(); catch (Exception e) e.printSt
18、ackTrace( );)Overridepublic Status process() throws EventDeliveryException Channel ch = getChannel();Transaction txn = ch.getTransaction();Event event = null; txn.begin(); try while (true) event = ch.take(); if (event != null) break;) )String rawbody = new String(event.getBody();/insert into tl(cont
19、ent) values(zhangsan*); String sql = insert into + tableName +(* + columnName+ )M+Mvalues(+rawbody+M)stmt.execute(sql); mit();return Status.READY; catch (Exception e) txn.rollback(); e printStackTrace(); return null;finally txn.close();)Overridepublic void configure(Context context) url = Hjdbc:mysq
20、l:/localhost:3306/test; user = root;password = 123456”; tableName = test*; columnName = content;)配置文件如下:al.sources = rlal.sinks = klal.channels = cl# # Describe/configure the sourceal.sources.rl.type = netcatal.sources.rl.bind = localhostal.sources.rl.port = 44444# # Describe the sinkal.sinks.kl.typ
21、e = flume.plugin.CustomSink# # Use a channel which buffers events in memory al.channels.cl.type = memory al.channels.cl.capacity = 1000al.channels.cl.transactionCapacity = 100 # Bind the source and sink to the channelal.sources.rl.channels = clal.sinks.kl.channel = cl改进:参数不写死,而是写在配置文件里,如:al.sinks.kl
22、.type = flume.plugin.CustomSinkal.sinks.kl.url =jdbc:mysql:/127.0.0.1:3306/testal.sinks.kl.tableName= testal, sinks.kl.user=rootal sinks.kl.password=123456al sinks.kl.column_name=content从配置文件中读取各种属性,并进行一些非空验证public void configure(Context context) columnName = context.getString(column_name);Precondit
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- 互联网 面试 真题附含 答案
![提示](https://www.deliwenku.com/images/bang_tan.gif)
限制150内