Cloud Computing Panel

Abstract/Agenda: 

This session invites four cloud computing leaders to discuss cloud computing from their projects to address relevant topics: 

  1. Hook Hua: Hybrid Cloud -- Application Amazon spot pricing and management
  2. Mike Little: AIST Cloud 
  3. Christopher Lynnes: EOSDIS and Cloud Computing
  4. Brian Wilson: Private Cloud and cluster computing

Relevant topics

1. The lastest from cloud offers. 

2. Cloud enable Earth Science projects. 

3. Pros and Cons of migrating to cloud comupting. 

4. Organizational strategies on cloud computing.

Notes: 

Hook

  • Key points: scaling, spot market but need resiliency(99.5% for 2000 nodes -> some nodes down per month), fault tolerant

  • Optimizing S3 Object Store Performance

    • S3 object key naming affects performance -> object partition

  • Market Maker:

  • Data lake: collocation; minimize data movement; maximize user services

    • hot data lake: long-term public cloud storage is expensive; object store data for hot data.

 

Mike Little:

  • The role of AIST: advanced information technologies for Earth science space-based and ground-based information systems; accessibility and utility of science data

  • Two Major Cloud Efforts: AMCE; Technology Developments which need cloud computing (SAR Science data processing: iISCE, ARIA series of projects)

  • AMCE: Overview

    • enable PI-led teams access to as much capability as possible

    • avoid financial management and procurement problems

    • minimize computer security problems, and management overhead costs

    • reduce cost of refactoring (OS, libraries, security, central authority)

    • avoid re-installation by SaaS

  • SAR SDP: various data products, science data processing(high volume, data quality, cloud with scalability)

 

Chris

  • Archive Volume increase fastly

  • EOSDIS Cloud Plans:

    • Object storage

    • Cloud Analytics Prototypes

  • Big questions:

    • how to supply data to all users on a non-discriminatory basis?

    • How to avoid storage lock-in?

    • How to attract users to cloud?

    • How to predict pricing?

    • How to migrate near archive data to cloud?

    • What should not go to cloud?

    • How to handle provisions and accounting of cycles?

    • Do we need new operations policies or procedures?

 

Wilson

  • advantages: Replicating services for High availability or Backup

  • Parallel Computing: Multi to many core; GPU daughter boards; compute clusters (MPI, OpenMP; Batch ma-reduce; in-memory map-reduce); Cloud computing

  • Comparison of Big Data Platforms (“A Survey on Platforms for Big Data Analytics”)

  • Data Locality/Movement Challenge

    • the data communication in cores, gpu, cpu, cloud

  • Cloud or Not?

    • Cloud compute VS. Buying hardware

    • Serving data and archive

    • Security

    • Modular, high-availability

 

Question:

  • Storage lock-in, what is the data policing?

    • relatively: some are big data; some are not;

  • Comment on cloud on education?

    • NASA cloud training;  

  • Data accessibility in cloud?

    • on-demand access

  • the capability of the cloud computing, is it the future direction?

    • not sure.

  • Risk of achieve data

    • one of biggest worrying; build own data centers;

    • public cloud & hybrid cloud

  • How to move the data to compute?

    • most cases are moving computing to data.

    • multi-copying to improve performance

Citation:
Yang, C.; Huang, T.; Cloud Computing Panel; Winter Meeting 2016. ESIP Commons , November 2015