WoTSF: A Framework for Searching in the Web of Things (WoT)

33 pages
128 views

Please download to get full document.

View again

of 33
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Share
Description
Paper Details M. Younan, S. Khattab, and R. Bahgat, "WoTSF: A Framework for Searching in the Web of Things," in INFOS2016: The 10th International Conference on Informatics and Systems, ACM, Cairo, Egypt, May, 2016. ABSTRACT - A key challenge in the emerging Web of Things (WoT) paradigm is how the human users and machines look for meaningful and readable information in huge and dynamic datasets in real-time, whereby the datasets are presented in different formats. This paper presents a technique to construct efficient, hierarchical web indices that are efficiently kept up-to-date. Also, a framework for searching in the WoT, namely WoTSF, is proposed and experimentally evaluated using a prototype. The proposed framework was shown to present a tradeoff between search speed and result accuracy as compared to the Dyser WoT search engine.
Transcript
  • 1. WoTSF: A Framework for Searching in the Web of Things M. Younan, S. Khattab, and R. Bahgat May 11th 2016 M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 2. 1 • Introduction 2 • Related Work 3 • The Architecture of the WoTSF 4 • The Implementation of the WoTSF 5 • Experimental Evaluation 6 • Conclusion and Future Work Agenda M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 3. INTRODUCTION M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 4. Introduction (WSN -> WoT) • Wireless Sensor Network (WSN) • Daily life integration of embedded devices – Convert things to Smart Things (SThs) – No.of devices reaches order of billions in 2020. – No.of users < no.of devices. • The Internet of Things (IoT). – Huge Sensory Data (data stream). • The Web of Things (WoT). – Current web tools and services. • Contribution Idea: – Searching in the WoT increases its popularity M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 5. Introduction • Features of the WoT (Searching) – Different formats. – Non-standardized naming – Huge and dynamic Sensory Data. – Daily life integration (normal user) • Simple query language • Real time queries • Interested in high level knowledge (summary) • Main Points: – Crawling • LWoTSEs’ Data –> following features of the WoT dataset presented in [1][2]. – Indexing • WoTSF: a Framework for searching in the WoT (High level indices) Crawling Indexing and Searching M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 6. RELATED WORK M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 7. Related Work (Selection)  Traditional Search engines o Google Optimizations [3] o AJAX Crawler [4]  DiscoWoT (Mayer and Dominque. 2011)[5].  Shodan SE (www. Shodanqh.com,2015)[6] .  Dyser SE (Ostermaier et al.- 2010)[7].  Zhang et al. (2015) present a framework for a distributed range-query search[8] M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 8. Dyser • Dyser uses periodic patterns for each sensor (L,O,W,P), where L: period length O: offset W: sensor output P: probability value E.g., (7-Days, 2, empty, 0.5) • Expansion of such periodic pattern is (K*L +O) Su Mo Tu We Th Fr Sa 17 18 19 20 21 22 23 24 25 26 27 28 29 30 For Example: If crawling process starts in 20-4, then Index record: (Occ., ID_01, 25-4-2016, empty, 0.5) # recorded patterns in Dyser index will be >= # SThs M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 9. Dyser Index-> WoTSF Index S01: (7-Days, 1, empty, 0.8) S02: (7-Days, 1, empty, 0.8) S03: (7-Days, 2, empty, 0.5) S04: (7-Days, 2, empty, 0.9) S05: (7-Days, 2, empty, 0.6) S06: (7-Days, 3, empty, 0.7) S07: (7-Days, 3, empty, 0.2) S08: (7-Days, 3, empty, 0.4) S09: (7-Days, 3, empty, 0.9) S10: (7-Days, 4, empty, 0.3) S11: (7-Days, 5, empty, 0.8) S12: (7-Days, 6, empty, 0.6) S13: (7-Days, 7, empty, 0.5) S14: (7-Days, 7, empty, 0.7) WoT01: (7-Days, 1, empty, 0.8, S01) WoT01: (7-Days, 1, empty, 0.8, S02) WoT01: (7-Days, 2, empty, 0.9, S04) WoT01: (7-Days, 3, empty, 0.9, S09) WoT01: (7-Days, 4, empty, 0.3, S10) WoT01: (7-Days, 5, empty, 0.8, S11) WoT01: (7-Days, 6, empty, 0.6, S12) WoT01: (7-Days, 7, empty, 0.7, S14) Using aggregation function: ‘Max.’ 1 2 3 4 5 6 7 WoTSF: High Level Index M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 10. THE PROPOSED WOTSF ARCHITECTURE M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 11. The WoTSF Architecture Query life cycle of the WoTSF Crawling (WoT) Google Opt. (server-root file) Save time: crawl less no.of pages parse less no. of pages (Union format) (High Level) M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 12. The WoTSF Architecture (Query Processing) Query q2 WoT_1 WoT_1 M.Ds WoT_2 M.Ds WoT_3 M.Ds DS_1 Ind. DS_2 Ind. DS_3 Ind. •Select WoTSEs APIs •Send sub-queries Key: sensor type b c d …… a q2 Master Indices •Generate sub-queries M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 13. The WoTSF Architecture (Index Structure) M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 14. The WoTSF Architecture (Index Structure) mimiMI SISI SI D D D D D S WWW WWW WWW A R S A R SThs-Level (individual WoTSEs) WoT-Level (WoTSF) Master Indices Secondary Indices Storage Dynamic Pages M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 15. The WoTSF Implementation (Ranking) • Sensor State probability  𝑆𝑆 𝑖 = 𝐾 ∗ 𝐶𝑆 𝑖 + 1 − 𝐾 𝑃𝑖 K-> constant (0:1), CS-> current state, P->Probability • Entity State probability  𝐸𝑆 𝑖 = 1 𝑛 𝑆𝑆 𝑗𝑛 𝑗=1 N-> number of sensors  𝐸𝑆(𝑖) = 𝐷(𝑗) ∗ 𝑆𝑆(𝑗)𝑛 𝑗=1 D(j) -> impact factor of sensor j on ES(i) M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 16. THE WOTSF IMPLEMENTATION M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 17. The WoTSF Implementation (Crawling) (b) Gateway Start Receive new_value Update prediction model (Sr) no.of. consecutive changes > M Calculate Entity State SrVal_Stability (Sr,new_value) = (cur_rec_time - last_rec_time(Sr))/interval no.of records > N Replace (oldest_value, new_value Add new_value Update (EoI, State) newChange (EoI, State) Listen to Sensors (a) Sensor Start Read Cur_Value Compare Cur_value with previous value (range) Send Cur_Value Change Wait Period (T) Yes Crawling Levels: – WoTSE – WoTSF M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 18. The WoTSF Implementation (Indexing) • The WoTSF is evaluated using Real dataset: We use Kth percentile (historical data) -> single record per time unit. e.g. Suppose, a temperature sensor reads a set of values {5, 20, 21, 20, 20, 21, 23, 20, 21, 23} then, average=19.4 50th = 20 M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 19. The WoTSF Implementation (Indexing) • The WoTSF is evaluated using random values: – prediction model (‘7-Days’) – SThs of type ‘occupancy’ in 10 WoT networks, -> 10,000 SThs – aggregation function ‘Max’ (empty room). – expanding the quadruple predictions The prediction model type -> periodical crawling processes. M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 20. EXPERIMENTAL EVALUATION M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 21. The WoTSF Implementation (Indexing) (a) (b) In case of distinct SThs values • Dyser Index (a) – Count -> Max reading (STh level) • WoTSF index (b) – Count -> Max reading (WoT level) Local search engine WoTSF M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 22. The WoTSF Implementation (Searching) • The WoTSF prototype: – implements autosuggestions like dyser. – filters buildings (static part) -> WoTSEs’ APIs – Evaluation is done on the dynamic part of the query (WoTSEs) using: • Building - Level (Master Indices) • SThs – Level (Secondary Indices) (a) (b) (c) M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 23. The WoTSF Evaluation (Index size) M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 24. Experimental Evaluation (WoTSF & Dyser) • Index Size M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 25. Experimental Evaluation (WoTSF & Dyser) • Processing Time - WoTSF indices (High Level) - Dyser Indices M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 26. Experimental Evaluation (WoTSF & Dyser) • Result Accuracy Accuracy = 𝐶𝑜𝑟𝑟𝑒𝑐𝑡 𝑉𝑎𝑙𝑢𝑒𝑠 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠) 𝑇𝑜𝑡𝑎𝑙 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 (𝑓𝑎𝑙𝑠𝑒 𝑎𝑛𝑑 𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠) Consistency = 𝐼𝑛𝑡𝑒𝑛𝑑𝑒𝑑 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠) 𝑇𝑜𝑡𝑎𝑙 𝐼𝑛𝑡𝑒𝑛𝑑𝑒𝑑 (𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+𝑓𝑎𝑙𝑠𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠) Building ID Room ID State Prediction Value 100 101 Empty 0.8 102 0.4 103 0.7 200 201 0.3 202 0.5 203 0.4 Each building represents a single WoT network and hosts an occupancy sensor in each room Search Engine Index Size WoTSF 2 Dyser 6 M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 27. The WoTSF Evaluation (Index size) Search Engine Number of Results all First K-Result = 4 Probability >= 50% WoTSF 2 2 2 DSE 6 4 3 WoTSF and DSE search results according to different query types. Search Engine Results Number of Results all First K-Result = 4 Probability >= 50% WoTSF Room 101 0.8    Room 202 0.5    DSE Room 101 0.8    Room 103 0.7    Room 202 0.5    Room 102 0.4   Room 203 0.4  Room 201 0.3  WoTSF and DSE searching results (list of rooms) according to different query types. M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 28. The WoTSF Evaluation (Summary) Criteria DSE WoTSF Granularity of master index Device (STh) level Network (e.g., building) level Case: Speed Search  All available results  More time for ranking.  top values per WoT network.  Faster search  Indices more up-to-date Case: Accurate Results  Indices and accuracy: based on prediction models.  Consumes more time  Indices: up-to-date.  Accuracy: high Pros  Less network overheads.  More consistent results  Small and semi-dynamic indices.  Less time for (crawling, parsing, indexing). Cons  Larger indices.  Harder to keep indices up-to- date.  More time for crawling, parsing, and indexing  Tradeoff between search speed and result accuracy M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 29. CONCLUSION AND FUTURE WORK M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 30. • The WoTSF:  works on the top of Dyser  supports simple query language  saves time consumed for crawling, parsing and indexing  builds High level indices  Supports two types of search: • Search in high level indices (speed search). • Search in secondary indices using LWoTSEs’ APIs (Accurate search)  increasing accuracy <-> indices are up-to-date  keeping individual LWoTSEs handle their network. Conclusion M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 31. Future Work • WoTSF has limitations such as: – It consumes more time in accurate search type. • Network overhead • Merging and Ranking results. – Scheduling the crawling processes • to balance network overhead and result accuracy – Using semantic technology: will be helpful for interoperability. – Considering other aggregation functions • LWoTSE has limitations such as: – Extracting prediction models from SThs historical data – Dynamic discovery. – Solving problem of using multiple formats partially (crawling) but not by meaning -> (ontologies) M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 32. References (Selected) [1] M. Younan, S. Khattab, and R. Bahgat, "An Integrated Testbed Environment for the Web of Things," in ICNS 2015 : The Eleventh International Conference on Networking and Services, ISBN: 978-1-61208-404-6, Rome, Italy, May, 2015, pp. 69-78. [2] M. Younan, S. Khattab, and R. Bahgat, "Evaluation of An Integrated Testbed Environment for the Web of Things ", in IntSys15v8n34 : International Journal On Advances in Intelligent Systems, v 8 n 3&4, December, 2015. [3] Google. (2010, Jan.) Search Engine Optimization (SEO) - Starter Guide. [4] P. Suganthan G C, "AJAX Crawler," in Data Science & Engineering (ICDSE), International Conference on. IEEE, Cochin, Kerala, July 2012, pp. 27-30. [5] S. Mayer, D. Guinard, "An Extensible Discovery Service for Smart Things," in in Proceedings of the 2nd International Workshop on the Web of Things (WoT 2011), ACM, San Francisco, CA, USA, June, 2011, pp. 7-12. [6] (2015, Jan.) shodan search engine. [Online]. www.shodanhq.com [7] B. Ostermaier, K. Romery, F. Mattern, M. Fahrmairz, and W. Kellererz, "A Real-Time Search Engine for the Web of Things," in The 2nd IEEE International Conference on the Internet of Things (IoT), Tokyo,Japan, November. 2010, pp. 1-8. [8] C. Zhang, T. Zhang, and M.Wang, "A Distributed Range Query Framework for the Internet of Things," in 18th International Conference on Intelligence in Next Generation Networks (ICIN 2015), IEEE 2015, ISBN 978-1-4799-1866-9, Paris, France, February, 2015, pp. 83-88. M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • 33. QUESTIONS ? THANKS M.Younan,S.Khattab,andR.Bahgat,"WoTSF:AFrameworkforSearchingintheWebof Things,"inINFOS2016:The10thInternationalConferenceonInformaticsandSystems, ACM,Cairo,Egypt,May,2016.
  • Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks
    SAVE OUR EARTH

    We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

    More details...

    Sign Now!

    We are very appreciated for your Prompt Action!

    x