Enterprise Risks in Chat GPT

Vimal Ramachandran


HLB HAMT - Accounting Firm in UAE

Phone:- +971 4 327 7775
Mobile:- +971 52 830 7998
WhatsApp:- +971 56 219 1607
Email:- dubai@hlbhamt.com

    Schedule a Consultation

    Generative AI (GenAI) has introduced new enterprise risks that must be properly evaluated in a business context. These risks include technological or procedural expansions of current hazards, legal and regulatory risks, and wholly new dangers. A reference table is provided as guidelines for organizations to assess these risks.

    Enterprises can reduce the negative impact of GenAI usage by opting out of having user prompt data used to train future models, accepting a data retention policy of 30 days, declaring a Risk Exception, or investigating an on-premises option.

    The authors seek to address data exposure issues related to GenAI and ChatGPT by using large language models (LLMs) to make user input and private data available to rivals. GenAI technologies may use user input for model development, but OpenAI’s stance on this is against it due to potential dangers and issues with biased models.

    High Risks

    A. Data Privacy and Confidentiality

    Private information and non-public enterprise The application of GenAI in the enterprise may result in the access and processing of sensitive information, intellectual property, source code, trade secrets, and other proprietary information. data, either direct user input or the API, including customer or private information and confidential information. This has previously been noted as a problem here.

    • Sending secret and private information outside of an organization’s network can lead to legal and compliance issues, as well as risks of information exposure, under CCPA, GDPR, and HIPAA.
    • GenAI technologies such as ChatGPT have been handed over to OpenAI, but the third-party SaaS is not yet integrated. This means that users will not be able to view data in real time, and future models may not use user input.
    • Corporate documentation states that OpenAI’s API is not maintained for more than 30 days and is opt-out by default, while ChatGPT is opt-in by default and charged a fee. However, provided information is vulnerable to storage and processing risks.

    B. Enterprise, SaaS, and Third-party Security

     Non-public enterprise data, third- and fourth-party software

    CISOs are concerned that data exchange with third parties will increase due to the widespread use of GenAI and integrations in third-party apps, leading to less predictable patterns.

    • Risks in the supply chain can be divided into three categories: relying on third-party security, relying on GenAI technologies, and relying on third-party quality assurance.
    • GenAI platforms may expose sensitive data such as customer data, financial information, and proprietary business information if their systems and infrastructure are not secure.
    • GenAI and ChatGPT are being integrated into third-party platforms such as Microsoft Azure OpenAPI and Office 365, creating a potential risk.
    • GenAI platforms are a high-value target for threat actors due to their limited number and increasing use.

    C. AI Behavioral Vulnerabilities 

     Model operator, non-public enterprise data

     Actors may use or cause models to be used in ways that reveal sensitive information about the model or cause the model to be damaged. do acts that are contrary to the design’s aims.

    • Attackers can use GenAI systems to circumvent AI behavior and make it execute unexpected tasks, which can have a negative influence on organizations and stakeholders.
    • Third-party applications with GenAI APIs can allow attackers to access user data, potentially allowing them to take actions on behalf of the user.
    • Injection attacks can be used to gain unauthorized access to business systems.

    D. Legal and Regulatory

    •  Regulatory Consideration: GenAI must be used following data privacy rules, such as GDPR, PIPEDA, and CCPA. Italy’s data protection regulator has temporarily prohibited the use of ChatGPT, and Germany is reviewing the issue.
    • Legal Consideration: GenAI used in consumer-facing communications can be regulated, resulting in legal or regulatory consequences. ChatGPT and chatbot services must be disclosed to clients to avoid potential legal action.

    Medium Risks

     Threat Actor Evolution

     A. Enterprise readiness, third parties

    Threat actors use GenAI for malevolent objectives, such as phishing attacks and social engineering. To address this, security awareness training and other social engineering measures must be re-evaluated, and controls must be mitigated.

    B. Organization’s legal exposure

     GenAI models are trained on a wide range of data, including an unknown amount of copyrighted and private content, causing ownership and licensing concerns between the organization and third parties.

    • GenAI models have been accused of utilizing material generated by others, which could lead to intellectual property violations and plagiarism, as the same material can be provided to multiple parties.
    • GenAI models may inadvertently infringe on copyrighted material without authorization from dataset owners.
    • The US Copyright Office has recommended denying copyright protection for GenAI works, allowing them to be freely used and copied.
    • GenAI may return code with proprietary content, such as GPL 3, which could be legally binding for organizations to distribute. It is important to consult with an attorney if any infractions occur.
    • Policies should be based on current intellectual property concepts.

    C. Insecure Code Generation

    • Software development projects and developers GenAI-generated code can be used without sufficient security audit or review, leading to the deployment of insecure programs in different systems and as “ground truth” for future model learning.
    • Organization’s brand and reputation GenAI’s output can be delivered with significant reputational risks, including erroneous, damaging, biased, or humiliating content, as well as safety concerns such as doxxing or hate speech.
      • The current generation of GenAI models has been seen to provide erroneous, inaccurate, incorrect, and deceptive data.
      • Using AI outputs without validation can lead to inaccurate assertions and facts, which can result in legal consequences such as libel.

    Low Risks

    A. Software Security Vulnerabilities

    Non-public enterprise data and system integrity

    • GenAI apps must be updated and secured with proper controls against traditional software vulnerabilities and their interaction with developing AI vulnerabilities.
    • GenAI systems are vulnerable to software vulnerabilities and AI flaws, increasing the risk.
    • Attackers can exploit flaws in front-end models to manipulate back-end models, triggering SQL injections when appended to model output.

    B. Availability, Performance, and Costs

    • Enterprise systems’ Resilience

      GenAI and OpenAI can present operational risks such as system downtime, performance, and failure. User mistakes must be included in threat modeling and architectural planning, and backup and disaster recovery methods are required. Operating an LLM can be costly, with each response costing “single digit cents” according to OpenAI.

    • Regulatory Compliance

      AI is gaining traction due to ethical standards for safety, security, fairness, transparency, explain ability, and general responsibility. Legal and regulatory structures are already incorporating these principles, so it is important to screen for potential impacts and consider mitigation measures. CISOs can also influence rules and educate authorities. ESG issues should also be evaluated.

    Get in touch with our digital transformation service consultants to know more about enterprise risks in Chat GPT!




    Would you like to rate us on Google?         


    Varun R Chandra
    Varun R Chandra
    07:38 25 Apr 21
    HLB Hamt is highly recommended! They have the quality of services that would satisfy your needs. I'm glad I found them.
    Giuseppe Assi
    Giuseppe Assi
    06:20 31 Jan 21
    Mohamed Sheriff
    Mohamed Sheriff
    16:16 28 Jun 20
    Experience with HLB Hamt in audit experience is great pleasure. They deploy well qualified accounts team for audits.... And the amount of checking they perform give the management of the company confidence that internal controls are well in place. We are glad to see they have high standard of audit practices.read more
    218141 JOSEPH THOMAS
    218141 JOSEPH THOMAS
    16:20 23 Jun 20
    Professionally managed firm with commitment to the engagements taken. My company and its group are assigning our... requirements for last 12 years to HLB and we don’t have any negative remarks on any assignments.read more
    07:14 23 Jun 20
    In my opinion, HLB Hamt is one of the best professional firms in UAE after big4. Excellent client service, professional... and competent staff and timely delivery are their strength. Over the years they have developed the infrastructure and skill set to handle multiple verticals ie, audit, tax services, consulting, IT support etc.Wishing HLB Hamt team all the very best.read more
    08:28 21 Jun 20
    Thank you Team HLB hamt...Your internal auditing team put the good amount of time and inputs in helping the... organisation making considerable improvements throughout the organisational working structure and helps in building the better business.Thanks.read more
    Maharajan Subramaniam
    Maharajan Subramaniam
    18:17 06 Jun 20
    High-Performing IT Teams, I'm impressed with the knowledge level of their deputed staff, audit procedures, and... checklist adopted. Mr. Vimal especially understand the requirement and provide solution not only technical feasibility and also considered budgeting. We’re confident in their abilities.read more
    Mohammed L
    Mohammed L
    08:07 05 Jun 20
    We have been working closely with them on payroll services for quite sometime.They provide very professional... services,experts in domain and excellent support. Would recommend them.Cheers.read more
    06:32 04 Jun 20
    Very professional team. Vimal has always given us good solution advice for various Software and IT Infra projects. We... are very Extremely satisfied working with him and his team. Wishing them success.read more
    Basil Daniells
    Basil Daniells
    05:55 03 Jun 20
    I have worked with HLB as a partner of Sage for about 2 years, they were dedicated to the success of the partnership... and kept aligned with the business plan that was agreed between our two companies. They are always ready to go the extra mile for a customer and don’t shy away from challenging circumstances. Keep up the good work and wish you all the success in the future.read more
    Amar Kashyap
    Amar Kashyap
    19:28 02 Jun 20
    Team of well qualified and organised professionals. Their reports are not based on numbers first they will understand... their client and their business. They are highly recommended and trust worthy.read more
    Radz de Jesus
    Radz de Jesus
    16:03 02 Jun 20
    We're glad to have found HLB Hamt to conduct the IT audit and risk assessment for our organization. We have learned a... lot from their team during that first project. And since then, we've continued to be in touch with them. Mr. Vimal especially is very good at presenting technical concepts in simple yet interesting ways that non-IT and business people alike would easily catch. We are very happy with the professionalism and quality of service they provide. They leave their clients nothing short of satisfied. Highly recommended.read more
    Next Reviews

    Get in touch

    Whatever your question our team will point you in the right direction

    Start the conversation
    Get in touch

    Share to:

    Copy link:

    Copied to clipboard Copy