Saturday, December 22, 2018

Big data

Company
- acxiom
- epsilon
- datalogix

Spatial data management
- eg countries, cities
- based on database management systems that have abstract data type functionality, types:
  • Postgresql geometric data types
- indexing and query processing
- spatial data warehousing
  • Multidimensional analysis - Online analytical processing 
  • Aggregation - distributive, algebraic, holistic
- spatial data mining
  • Clustering
  • Dbscan algorithm - density based
  • Optics algorithm
  • Co location data mining 
  • classification
  • dependency analysis
  • deviation and outlier analysis
  • Trend detection
  • generalization and characterization
collaborative mapping
OpenStreetMap is a collaborative project to create a free editable map of the world. Rather than the map itself, the data generated by the project is considered its primary output. The creation and growth of OSM has been motivated by restrictions on use or availability of map information across much of the world, and the advent of inexpensive portable satellite navigation devices.[6] OSM is considered a prominent example of volunteered geographic informationCreated by Steve Coast in the UK in 2004, it was inspired by the success of Wikipedia[7] and the predominance of proprietary map data in the UK and elsewhere.[8] Since then, it has grown to over two million registered users,[9] who can collect data using manual survey, GPS devices, aerial photography, and other free sources. This crowdsourced data is then made available under the Open Database License. The site is supported by the OpenStreetMap Foundation, a non-profit organisation registered in England and WalesThe data from OSM is available for use in both traditional applications, like its usage by Facebook, Craigslist, OsmAnd, Geocaching, MapQuestOpen, JMP statistical software, and Foursquare to replace Google Maps, and more unusual roles like replacing the default data included with GPS receivers.[10] OpenStreetMap data has been favourably compared with proprietary datasources,[11] although in 2009 data quality varied across the world.

  • Coast grew up in Walderslade and London, England. Coast interned at Wolfram Research before studying computing science at University College London (UCL). In July 2004, he founded the OpenStreetMap project (OSM). Coast set up Z.X.V. Ltd. with Nick Black, Tom Carden and Ben Gimpert as a technology consultancy in 2005. In 2008 this became CloudMade after investment by Nikolaj Nyholm and Sunstone Capital (company's website claims it was established in 2007[4]).[5] He resigned from CloudMade in October 2010,[6] although he remained a shareholder.[citation neededOn 23 November 2010, Coast announced that he had accepted a position as Principal Architect at Microsoft's Bing Mobile. On 3 September 2013, Coast wrote on his blog that he had started to work for TeleNav, taking care of OSM development for the company's Scout navigator. In March 2014, Coast became an advisor to Auth0, an identity-as-a-service provider. In November 2015, Coast published "The Book of OSM". The book contains 15 interviews conducted by Coast with various users who had participated in the project since its beginning. In January 2016, he stepped down from full-time work at TeleNav[11] and started his work as a board advisor in Navmii.[12] In March 2016 he started working as Chief Evangelist for what3words.[13][14] In May 2016 he became an advisor in MapJam.[15] In fall 2017, he joined DigitalGlobe.[16] In spring 2019 he joined TomTom as a vice president in the Maps department.
  • [action asia jan/feb2018] since the inception of OSM, the program has received backing form UCL, Imperial college london and a plethora of ict companies like bytemark and delta telecom; AR game pokemon go switched to its platform form google maps in early december (2017?)


Data trading
- http://www.scmp.com/business/companies/article/2119627/shanghai-sets-data-trading-platform-latest-push-boost-citys
- economist 29feb2020 "data, data everywhere" new thinking on how to value on of the world's most precious resources

alternative data vendors
https://www.ft.com/content/d86ad460-8802-11e7-bf50-e1c239b45787 In recent years there has been a proliferation of new “alternative data vendors” that trawl through vast pools of digital information and sell it to investment groups desperate for an edge in markets. These vendors often scoop up the digital “exhaust” that people, companies and countries throw out through the normal course of their business and turn it into valuable intelligence. For example, hints of Under Armour’s downturn could have been detected in a decline in job listings on its website, the internal rating of its chief executive by employees on Glassdoor, the recruitment site, or a dip in the average price of clothes on its website. But this is just the tip of the alternative data iceberg, and investors are waking up to the fact.

blockchain applications
- https://beta.scmp.com/business/companies/article/2179106/what-extent-can-blockchain-help-fight-slavery-global-supply-chain


Safe Harbour ruling
- http://www.ft.com/cms/s/2/7544e716-6b87-11e5-aca9-d87542bf8673.html For the past 15 years, a little-known agreement called “safe harbour” has quietly enabled the likes of FacebookAmazon and Google to become wildly popular in Europe. But the agreement, which allows companies to transfer customer data from the EU to the US, has in effect been scuppered by the European Court of Justice in a judgment released on Tuesday. The ECJ, endorsing a ruling last month by a top adviser to the court, concluded that the “safe harbour” agreement was invalid because it prevented data protection authorities from intervening to protect citizens who claimed their right to privacy had been breached. So what is the agreement, how did it get into trouble and why does it matter?
Why do companies transfer data?
Data has to be stored somewhere. Everything we put online — from pictures of nephews to credit card details — eventually finds its way on to a server.
Since the internet is dominated by US companies, these servers tend to be based in the US. Transferring data across the Atlantic is generally a quicker and cheaper option than building a wholly independent set-up in Europe, especially for younger companies.
What is safe harbour?
Safe harbour is an agreement between the US and the European Commission that lets businesses transfer data on European citizens across the Atlantic.
The EU’s rules on data protection make it illegal to transfer personal details to any country that does not meet the bloc’s privacy standards.
Safe harbour, which was set up in 2000, provides a shortcut for companies operating in the US. If they agree to meet certain rules — such as notifying customers when their information is collected and used — they can then transfer data from the EU to the US.
So why is it important?
Roughly 4,400 companies use safe harbour, including some of the world’s biggest technology groups such as Facebook and Amazon.
Without safe harbour, data transfers across the Atlantic would become much more cumbersome for both big and small businesses. Companies could be forced to operate effectively as separate entities in the US and the EU, according to the agreement’s defenders.
More broadly, data transfer is a key issue in the sprawling trade deal the EU and US are negotiating. The court’s decision is likely to add yet another degree of complexity to the Transatlantic Trade and Investment Partnership.
How did this legal challenge come about?
Max Schrems, a 27-year-old Austrian law student, took the Irish Data Protection Commissioner — which regulates Facebook — to court.
Mr Schrems argued that the regulator had failed to protect him from the mass internet surveillance by the US National Security Agency revealed in 2013 by Edward Snowden.


Privacy
- http://www.economist.com/news/science-and-technology/21660966-can-big-databases-be-kept-both-anonymous-and-useful-well-see-you-anon

data protection laws
- https://www.dlapiperdataprotection.com/#handbook/world-map-section

tech giants
- ft 7jun19 Microsoft pulls faces database

banks
- JPMorgan Chase, which has the biggest sales and trading businesses in the world, is launching a new Customer Relationship Management and analytics system so that its sales people can sell better, a person familiar with the project said. While CRM has been common place in other industries for decades, banks have been slow adapters. Their systems are notoriously poorly linked up, traders usually have to manually look up clients and may only then get some of the information the bank holds about that clients’ dealings and preferences. JPMorgan’s system works by linking clients’ phone calls to profiles of the customers’ previous trades and trading behaviours, using the same technology that call centres have employed for decades. https://www.ft.com/content/1eaf6436-e4a2-11e6-9645-c9357a75844a

Insurance
Insurers warned to use 'big data' responsibly ft 2feb15
- 有銀行家認為,香港銀行可參考美國銀行業界,利用大數據劃分全港各區客戶資料,從而調整定期存款息率,或透過分析客戶存款資料,針對不同客層,提供特別存息優惠。http://orientaldaily.on.cc/cnt/finance/20180529/00202_016.html

medical
16 日,深圳國家基因庫與 全球共享流感數據倡議組織(GISAID)達成 戰略性合作。雙方將圍繞呼吸道傳染病相關 病毒的基因數據管理、共享、分析等方面展 開全面合作,促進病毒數據的全球共享。國 家基因庫生命大數據平台成為 GISAID 的中 國首個正式授權平台。hkcd 17mar2020 a11
政府昨發表香港基因組醫學發展策略,提出數項建議,包括預留十二億元撥款,在六年內推行香港基因組計劃,冀將罕見病的診斷率由一成提升到三至四成,以及加強規管使用基因數據作保險及就業用途等。本月成立由政府全資擁有公司「香港基因組中心」協調推行計劃,聘請六十至一百人,年底前開展前期預備工作,而臨床工作及招募病人由香港兒童醫院、瑪麗醫院及威爾斯親王醫院負責,長遠希望建立大型數據庫,提升針對本地人口特有疾病的研究,及將數據應用於臨床治理及研究。投保人如測出屬某疾病高危人士或影響醫保保費,政府擬透過保險業界守則限制使用基因訊息作保險用途,參考國際慣例規定市民投保時毋須提供基因測序報告,但不包括保額逾一百萬元的危疾及逾五百萬元的人壽保險;政府亦會加強僱主及僱員就防止基因歧視的公眾教育。https://orientaldaily.on.cc/cnt/news/20200515/00176_034.html

Article
http://blogs.hbr.org/2013/09/nate-silver-on-finding-a-mentor-teaching-yourself-statistics-and-not-settling-in-your-career/
- http://www.economist.com/news/leaders/21616953-surveillance-advertising-industrys-new-business-model-privacy-needs-better
- http://forumblog.org/2014/09/whats-value-personal-data/?utm_content=buffera956b&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer
- http://www.economist.com/news/international/21629417-business-thriving-anonymous-internet-despite-efforts-law-enforcers
- http://www.economist.com/news/international/21632520-rich-countries-are-deluged-data-developing-ones-are-suffering-drought
- http://www.ft.com/intl/cms/s/0/7933792e-a2e6-11e4-9c06-00144feab7de.html financial companies have the option of using data-guzzling technologies that make the observation of shopping habits look downright primitive. A plethora of information gathered from social media, digital data brokers and online trails can be used to mathematically determine the creditworthiness of individuals, or to market products specifically targeted to them. The degree to which such algorithms are utilised by mainstream banks and credit card companies is unclear, as are their inputs, calculations and the resulting scores. While many types of data-driven algorithms have been criticised for opacity and intrusiveness, the use of digital scorecards in finance raises additional issues of fairness. Using such information to make predictions about borrowers can, critics say, become self-fulfilling, hardening the lines between the wealthy and poor by denying credit to those who are already associated with not having access to it.


knowledge platform
- http://www.bigdata-startups.com/

Counter surveillance
- http://www.slate.com/articles/technology/future_tense/2015/03/data_and_goliath_excerpt_the_best_ways_to_undermine_surveillance.html excerpt from Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World by Bruce Schneier, published by W. W. Norton & Co. Inc.

property ownership
- https://www.economist.com/news/leaders/21729745-and-threaten-property-rights-digital-age-how-digital-devices-challenge-nature How digital devices challenge the nature of ownership Privacy is also at risk. Users were appalled when it emerged that iRobot, a robotic vacuum cleaner, not only cleans the floor but creates a digital map of the home’s interior that can then be sold on to advertisers (though the manufacturer says it has no intention of doing so). After hackers discovered that a connected vibrator, called We-Vibe, was recording highly personal information about its owners, its maker, Standard Innovation, agreed in a settlement to pay customers and their lawyers up to $3.2m, with a maximum of $127 for each claim. And farmers complain that, if crisis strikes at the wrong time, John Deere’s requirement that they use only authorised software, which funnels them to repair shops that may be miles away, can be commercially devastating. Some are sidestepping the curbs with hacked software from eastern Europe.

business model / revenue generation
- scmp 26nov18 airlines race to cash in big data jackpot

personal data
- 百佳超級市場因未獲資料當事人同意而將其個人資料作直接促銷,違反《個人資料(私隱)條例》相關條文,昨在屯門裁判法院被判罰款三千元,成為《條例》自一三年修訂以來首宗被定罪的案例。http://orientaldaily.on.cc/cnt/news/20180103/00176_064.html

內地網絡安全公司「默安科技」的首席技術官(CTO)魏興國,前日在微博上發文披露,微博發生用戶數據洩漏事件。有關貼文隨後遭刪除,但內媒記者隨後發現,一些外國網站交易平台出現相關數據買賣,繳費後可通過微博帳號查詢到用戶的手機號碼,甚至詳細的個人資料。https://orientaldaily.on.cc/cnt/china_world/20200321/00178_006.html

Eu research projects
-http://move-cost.info
-http://modap.org
-http://www.geopkdd.eu
-http://www.uhasselt.be/datasim
-http://www.seek-project.eu

australia
- 澳洲競爭與消費者委員會(ACCC)周五宣布,將收緊對美國科網巨企Facebook(Fb)及Google的私隱及廣告政策限制。ACCC計劃建立全球首個針對科技企業的監察部門,打擊不當儲存或使用用戶數據的科企。https://orientaldaily.on.cc/cnt/china_world/20190727/00180_011.html

china
疫情爆發之下,中國零售、餐飲與旅遊市場首當其衝,估計這些行業在春節假期直接經濟損失超逾一萬億元人民幣,可能拖累今年首季中國經濟減速一至兩個百分點。不過,隨着線上辦公、教育與醫療等平台與軟件在疫情期間廣泛應用,用戶大幅增加,成為內地居民在疫情期間提供工作與生活上便利的應用工具,將逐步顛覆傳統辦公與教育運作模式,大大促進中國數字經濟發展。 據了解,華為的WeLink、阿里巴巴的釘釘、騰訊的企業微信等科技巨企紛紛開放在家辦公、網上教室等軟件與App,用戶數目一下子暴增,其中釘釘錄得單日線上辦公用戶接近兩億。http://paper.takungpao.com/resfile/PDF/20200205/PDF/a20_screen.pdf
記者12日從珠海市發改局獲悉,投資6.55億元人民幣的衛星大數據產業園,列入《廣東省2020年重點建設項目計劃》,擬今年落成啟用,將成為大灣區首個衛星大數據產業園。由珠海歐比特宇航科技自主研發和運營的珠海一號衛星星座,已完成12顆衛星組網,其中8顆高光譜衛星具備2.5天覆蓋全球一遍的服務能力。疫情期間,珠海一號不間斷收集武漢及周邊地區光譜數據,為防控新冠肺炎疫情助力。據了解,珠海一號衛星星座項目目前正加快建設,今年內將建成「衛星大數據產業園」,包括衛星運營中心、衛星大數據處理中心、衛星大數據應用示範中心,設計運營4顆視頻微納衛星、10顆高光譜微納衛星、2顆雷達微納衛星。http://paper.takungpao.com/resfile/PDF/20200313/PDF/a14_screen.pdf

Hong Kong
- http://www.pland.gov.hk/pland_en/info_serv/statistic/tpedm14.html The Territorial Population and Employment Data Matrix (TPEDM) is compiled by the Planning Department (PlanD) and updated regularly in about every two to three years to tie in with the updated population projections.
衞生防護中心傳染病處主任張竹君表示,衛生署已和醫管局開會,研究如何加強化驗所監測,以便更早發現較輕微個案。為令市民可掌握最新疫情發展,衛生署已連同醫管局、資訊科技辦公室等,於本周一晚推出「新型冠狀病毒感染本地情況互動地圖」 (https://chp-dashboard.geodata.gov.hk/nia/zh.html),網站以地圖和列表形式,展示確診個案、死亡個案、個案詳情、確診個案曾逗留的地點及逗留日期等信息,供市民參考。http://pdf.wenweipo.com/2020/02/05/a02-0205.pdf


1 comment:

  1. Apply Online for Business Loans

    *Commercial Loans.
    *Personal Loans.
    *Business Loans.
    *Investments Loans.
    *Development Loans.
    *Acquisition Loans .
    *Construction loans.
    *Credit Card Clearance Loan
    *Debt Consolidation Loan
    *Business Loans And many More:

    Quick Approval. Highlights: Providing Flexible Loans,Email: capitalfunding454@gmail.com

    ReplyDelete