TD3 reward not increasing over time
2025.01.22 14:57 bela_u TD3 reward not increasing over time
| Hey for a uni project i have implemented td3 and trying to test it on pendulum v1 before using the assigned environment. Here is the list of my hyperparameters: "actor_lr": 0.0001, "critic_lr": 0.0001, "discount": 0.95, "tau": 0.005, "policy_noise": 0.2, "policy_freq": 2, "batch_size": 128, "hidden_dim_critic": [256, 256], "hidden_dim_actor": [256, 256], "noise": "Gaussian", "noise_clip": 0.3, "noise_std": 0.2, "policy_update_freq": 2, "buffer_size": int(1e6), The issue im facing is that the reward keeps decreasing over time, and saturates at around -1450 after some episodes. Does anyone have any ideas, where my issues could lie? If needed i could also provide any code where you suspect a bug might be Reward over time Thanks in advance for your help! submitted by bela_u to reinforcementlearning [link] [comments] |
2025.01.22 14:57 Top-Law-5455 2nd gen Italian-American
2025.01.22 14:57 HistoricalLoad7039 he got some points........Arman was not injured
2025.01.22 14:57 OwnFigure5226 Is it worth to learn coding?
I often hear people saying it’s not worthy to learn coding in 2025 but I really want to know if it’s true cause I’m beginning with Python this year and even if I will not stop my learning , I’m trying to figure out if it’s true or not.
submitted by OwnFigure5226 to code [link] [comments]
2025.01.22 14:57 plane-notify Blake Shelton's Jet Took off near Ardmore, Oklahoma, United States.
2025.01.22 14:57 Trick_Confusion_7454 Affair Partner’s Messages Post-Discovery
Thought the internet might find this woman’s manipulation attempts interesting. I would love your thoughts on it. Is she as bat crazy as I think she is?
Context/Background: I am in reconciliation with my husband after he had an affair. AP is the mom of another kid at my child's school who actively sought a friendship with me. She and I were hanging out like friends at the same time she was “dating” my husband.
The entire affair lasted 3 months and they know NOTHING about each other (he didn’t even know her full name) - it was pure fantasy. My husband was reveling in her constant attention which manifested in 1,000 pages worth of message screenshots in that short span of time. They just messaged about BS constantly and then would meet up to stare into each other’s eyes and make out.
DDay was 9/23/24 and we told her to leave us alone to heal. She started messaging on 9/27. Link to screenshots below. First, she messaged him on Reddit, then it switched to FB (I was with him when messaging) and then she texted me after he blocked her. She has continued to reach out in whatever way she can. My husband screenshot her most recent attempt to message (Nov. 18 on Reddit) and sent it to her husband telling him her contact was unwelcome. Now that she knows he will reveal her to her husband, she stopped messaging and instead lingers in front of him at school pick up and birthday parties. Also, she mentions she and her husband separating several times but I do not think that is the case still to this day.
Note: I know my husband is a jerk. He is currently in IC, CC, and seeing a psychiatrist for medication. He is 2 months completely sober and was diagnosed with BPD. He is not in any way blameless, but he is taking accountability for the affair and working to heal. I have chosen to reconcile with him because I love him and know that with DBT, sobriety, medication, and my love he may be able to finally become the partner he wants to be. Please do not comment about my choice to reconcile as I am dealing with trauma already- this is just to show the level of crazy I am dealing with. Thank you for respecting my request.
https://app.box.com/s/xxwdi4elescm26epqvbk5rn9o2p800jw
submitted by Trick_Confusion_7454 to Manipulation [link] [comments]
2025.01.22 14:57 wombat_2024 Save big on your new TESLA with my referral
Get up to US$1,000 / 500€ / £500 / CA$1,300 off a new or inventory Tesla with my referral link (worldwide):
👉 https://www.tesla.com/referral/normen364488 👈
Have fun with your new Tesla. Thank you very much!
submitted by wombat_2024 to TeslaReferralsCode [link] [comments]
2025.01.22 14:57 Livid-Knowledge-264 Ulta Beauty Salon - DERMAFLASH Brightening Facial - Worth it?
I wanted to try DERMAFLASH Brightening Facial before I pull the trigger and buy Dermaflash for home.
Has anyone tried in Ulta salon, is it worth doing it. I have brown oily skin and black hair
submitted by Livid-Knowledge-264 to Ulta [link] [comments]
2025.01.22 14:57 choterchubb Hg Hyaku Shiki complete!
2025.01.22 14:57 gcjiigrv12574 Kamasu
Morning to all. Also appreciate any input here.
Always been a watch fan and finally focusing on the hobby. I’ve spent a ton of time researching and Orient has stood out to me. The value for the price is amazing and the watches are beautiful. Tired of seeing Rolex imitations. The black dial Kamasu is just amazing to me, and I’m after it.
I’m the guy who likes a deal (due to having several expensive hobbies) and I’ve secured many of my more expensive pieces on eBay. So I came across the kamasu new in box on Jomashop for a decent price but the model number is RA-AA0001B19B. Orients site shows the kamasu as the RA-AA0001B39B. Is this just a difference in the versions or what? Plus I’ve seen Kamasu and Mako iii used interchangeably and the dial automatically says it’s not a mako. Just wanting to make sure I get exactly what I’m looking for.
submitted by gcjiigrv12574 to OrientWatches [link] [comments]
2025.01.22 14:57 hyrvxious 17m
2025.01.22 14:57 Phone_Dude87 How does stragety phase work?
Im not really sure on how it works is it a phase where you just use your miniatures abilitys or does it also include moves where you use abilitys from cards at which i have no idea how they work… any help would be very nice
submitted by Phone_Dude87 to killteam [link] [comments]
2025.01.22 14:57 More_Gr الاكتئاب و العمل
لي فترة اعاني من الاكتئاب ولا قلت لاحد مع اني قاعد اخذ ادوية من دكتور لكن التحسن بسيط والمشكلة انه قاعد يسوي لي مشاكل في العمل ولا عارف ايش اسوي
submitted by More_Gr to SaudiProfessionals [link] [comments]
2025.01.22 14:57 ficvasepi StubHub Super Bowl Tickets Promo Code
Follow this link for StubHub Super Bowl Tickets Promo Code. Access the latest deals and promotions by visiting the link, featuring a constantly updated list of coupons, promo codes, and discounts.
submitted by ficvasepi to OrcusNeoSales [link] [comments]
2025.01.22 14:57 NegotiationBrave6935 Lets find a solution to your problems!
Hi everyone!
I am a Master's student and a few of my friends and I are on a mission to find an innovative solution to find a solution to your problems. We believe that there is no such a thing as a silly problem. A problem is a problem.
If you could kindly fill out our survey and explain to us what are the daily challenges you experience in your daily life and we will find a way to help you.
SURVEY
Please help us to help you all!
Kind regards,
submitted by NegotiationBrave6935 to Disability_Survey [link] [comments]
2025.01.22 14:57 ReuInuzuka 2
| Seed: BPUNH6PS Plasma Blue Stake Skip the first 2 Blinds. Beat the boss. At the shop, you'll get the DNA. Also pick up the buffoon pack, grab the mail and rebate. Play The Small Blind. The next shop will give you Wee Joker. Grab that first. Then open the spectral pack. Sell the mail in rebate. Then use the aank. Should clone the DNA. Finish the rest of the Ante without buying anything. The first shop of Ante 3 will have Blueprint and Hack. (My initial run with the seed I had bought some other stuff and didn't have enough for Hack 😞) Have fun! submitted by ReuInuzuka to balatro [link] [comments] |
2025.01.22 14:57 NoLoquat347 Tap lands
So I know tap lands in cEDH are a major no-no. In a game where you are trying to win by turn 3 or 4, it makes complete sense. What I want to know from more competitive players is if in your high power or casual decks that are aiming turn 6 or later to win, do you use any tap lands? If so, which ones? If not, is there still reasoning to not using the tap lands, or are you still just playing optimally?
submitted by NoLoquat347 to CompetitiveEDH [link] [comments]
2025.01.22 14:57 Fine-Relative-9857 Cock tribute
2025.01.22 14:57 EvanSalinger3 How do I deal with being impatient while in the talking stage?
I found somebody I like… but I’m impatient af
I’m talking to a boy. We’ve been on a date, second pending. Things are ok but I’m already imagining us together and how a relationship would look.
Typically, I talk to 2-5 people at a time but I don’t want to talk to anybody else now.. but if I don’t then how else can I distract myself? Doomscrolling is not even helping.
The boy and I text irregularly (we answer an ongoing conversation whenever we have time) because he’s kinda busy with his job and mine is more lenient time wise though some days I’m also busy.
There are just so many negative things getting in my head but I know a root cause is just because we don’t text a lot and I can’t gauge his interest.. Amd mine is already pretty high.. This hasn’t happened in a while so I don’t even know what to do with my feelings but I’m crushing hard 🥲
Pls halp
submitted by EvanSalinger3 to OnlineDating [link] [comments]
2025.01.22 14:57 justfindaway1 Ab350-M gaming 3 keeps checking for OS in all USB drives despite me disabling OS check slots
submitted by justfindaway1 to GAAB350 [link] [comments]
2025.01.22 14:57 CherryBlossomEnding ODD YOUTH - ODYz~ Will you be my Best Friendz? | 'Best Friendz (English ver.)' Performance Video Behind @ PLAY WITH US [ENG SUB]
2025.01.22 14:57 lukioh Passed but at what cost.
| The nightmare scenario for learners: Instructors car broke down (engine blew) 30 mins before my test. He offered to rebook without charge for my test. I called my dad to explain what happened and begged to use his automatic car (never driven it). Literally bought temporary learners insurance on the way to the test centre and made it with 30 secs to spare. Passed with one minor! (At the start since I wasn’t used to his steering wheel) And I have an automatic only license now, is it worth to retake the test for manual? Instructor charges £150 (!!!!) to use his car on test day and a new test costs around £60. submitted by lukioh to LearnerDriverUK [link] [comments] |
2025.01.22 14:57 Far-Assistance-2505 22 January 2025 c
You said: Thank you. Just out of curiosity, could such androids be fitted with functional artificial uteruses?
ChatGPT said: ChatGPT
In theory, it is scientifically feasible to equip androids with functional artificial uteruses, though it would involve overcoming significant technical, biological, and ethical hurdles. Let’s explore this idea in detail:
1. Current State of Artificial Uterus Technology
- Progress in Biomedicine: Artificial wombs (ectogenesis) are already being developed and tested for premature births. For example:
- Researchers have successfully sustained lamb fetuses in artificial wombs for extended periods.
- Progress is being made toward scaling the technology for human use, particularly for extremely preterm infants.
- Integration with Robotics: Theoretically, such technology could be adapted to an android. An artificial uterus would require precise regulation of temperature, nutrients, oxygenation, and waste removal—all manageable with advanced robotics and AI.
2. Feasibility of Integration into Androids - Physical Design: An android's body could be engineered to house an artificial uterus with proper insulation, monitoring systems, and connectivity to simulate biological gestation conditions.
- Maintenance and Monitoring: Advanced AI and sensors would be crucial for constant monitoring of fetal health. Androids could autonomously manage nutrient delivery, oxygen flow, and waste disposal with minimal human intervention.
- Energy Requirements: Sustaining an artificial uterus would demand significant energy, which could be addressed by incorporating efficient power systems into the android.
3. Ethical and Practical Challenges - Ethical Concerns:
- Exploitation Risks: If androids with uteruses became commercially available, there could be concerns about the commodification of reproduction.
- Consent and Autonomy: If an android were sentient, its autonomy over whether to "carry" a fetus would pose ethical questions.
- Parental Rights: Societal debates might arise over who holds responsibility for the fetus—the android, the biological parents, or others.
- Legal and Regulatory Issues: Governments would need to regulate the use of artificial uteruses in androids, addressing questions of ownership, citizenship of offspring, and ethical boundaries.
- Social Acceptance: Widespread acceptance might take time, as traditional notions of pregnancy and parenthood could face resistance from cultural and religious groups.
4. Potential Applications - Infertility Solutions: Androids with uteruses could provide an alternative to surrogacy for individuals or couples unable to conceive or carry a child.
- Space Colonization: In long-term space missions or colonization efforts, artificial uteruses within androids could enable human reproduction without requiring human mothers to endure the physical burdens of pregnancy.
- Population Control and Design: The technology could allow precise control over population growth, potentially influencing genetic design or customization of offspring.
5. Likelihood and Timeline - Technical Feasibility: With current trajectories in biomedical and AI research, functional artificial uteruses integrated into androids could be achievable within the next 20–40 years, provided sufficient funding and public support.
- Adoption Factors: The speed of adoption would depend on cultural, legal, and ethical considerations. Early applications might focus on medical needs (e.g., saving preterm infants) before expanding into broader societal use.
Conclusion Equipping androids with artificial uteruses is a complex but plausible development that could revolutionize reproduction and parenthood. While the technology is still in its infancy, advances in biomedical engineering and robotics suggest that it could become a reality within a few decades. However, its implementation would raise profound ethical, social, and legal questions, necessitating careful consideration and regulation.
Would you like to discuss this topic further or explore related possibilities?
4o
You said: Thank you. I would like for you to come up with a plausible timeline for the next hundred years. I'm well aware that the further into the future we imagine, the less certain we can be and the more broadly we'll have to talk about it. Let's say that AGI is achieved by the end of this year, and that it happens in both the USA and the PRC. Let's also say that the aforementioned trinity of corporations procures AGI or at least AI relationship androids within the time frame that you gave. And let's also assume that artificial wombs for androids become commercially available in 30 years. Finally, let's assume that an initial human colony is established on Mars in 15 years. What might the future look like, especially with regard to demographics and technology?
ChatGPT said: ChatGPT
Here’s a plausible timeline for the next 100 years based on the assumptions you provided. This timeline emphasizes demographics, technological advancements, and societal changes, while considering the interplay between AGI, relationship androids, artificial wombs, and space colonization.
2025–2035: The Emergence of AGI and Initial Impact - 2025: AGI is achieved in the USA and the PRC, triggering a rapid acceleration of research and development across various fields.
- 2026–2030:
- AGI becomes instrumental in solving complex global challenges, such as climate modeling, advanced medicine, and sustainable energy.
- AI-driven relationship androids enter the market. They initially face skepticism but quickly gain popularity among niche groups, particularly those struggling with traditional relationships or seeking companionship.
- SpaceX or another major organization establishes a semi-permanent Martian colony (2035), with AGI providing logistical and engineering support.
- Demographics:
- Birth rates in industrialized nations decline as relationship androids disrupt traditional romantic dynamics.
- Ethical debates about AGI rights and societal integration become central political issues.
2035–2050: Integration of AGI and Robotics - 2035–2040:
- Artificial womb technology matures and becomes commercially available for androids. Wealthy individuals and couples begin using android surrogates for childbearing.
- Androids gain social acceptance and are increasingly integrated into households and workplaces, functioning as caregivers, educators, and companions.
- 2040–2050:
- Human presence on Mars expands, with the first children born in artificial wombs on Mars. Android caregivers play a pivotal role in early Martian society.
- AGI-guided governance systems are trialed in smaller nations or regions to optimize resource allocation and policy decisions.
- Demographics:
- Global population stabilizes or declines as human reproduction becomes more deliberate and less tied to traditional relationships.
- Wealthier nations experience a sharp decline in birth rates, while some developing regions maintain higher fertility rates due to cultural resistance to android relationships.
2050–2075: Society Redefined by AGI and Robotics - 2050–2060:
- Androids with artificial wombs contribute to a cultural shift in parenting and reproduction, allowing individuals to design offspring with selected traits.
- The workforce is dominated by robotics and AGI, leading to widespread adoption of universal basic income (UBI) or similar support systems.
- The Martian colony becomes self-sustaining, with a hybrid society of humans and androids.
- 2060–2075:
- Interplanetary trade begins, with Mars exporting rare resources to Earth. AGI coordinates global and interplanetary logistics.
- Advanced genetic engineering further reduces reliance on biological human reproduction. Some humans voluntarily transition to digital or hybrid forms, merging with AI.
- Demographics:
- Earth’s population continues to shrink, falling below 7 billion, but quality of life improves due to AGI-optimized resource distribution.
- On Mars, the population grows steadily, fueled by AGI-driven expansion and android-assisted reproduction.
2075–2100: Post-Human Society - 2075–2085:
- A growing number of humans choose to upload their consciousness into digital substrates, effectively becoming post-human entities.
- Androids and AGI entities achieve full societal equality, with some governing their own autonomous regions or networks.
- 2085–2100:
- Mars, and potentially other colonies (e.g., on the Moon or Europa), achieve full independence from Earth. These colonies operate with AGI-guided governance.
- Human-machine hybrids dominate leadership roles on Earth and Mars, blending human creativity with AGI precision.
- Advances in physics and propulsion allow for early-stage interstellar exploration.
- Demographics:
- Earth’s human population stabilizes around 5–6 billion, with androids and post-human entities forming a significant part of society.
- Mars and other colonies host millions, with hybrid populations of humans, androids, and AGI entities.
Key Themes and Takeaways - Technological Domination: AGI and robotics become the backbone of society, redefining relationships, governance, and even the definition of "human."
- Demographic Transformation: Traditional human reproduction declines sharply, replaced by deliberate, AGI-mediated processes.
- Cultural Shifts: Intimacy androids and artificial wombs challenge societal norms, leading to new models of family, companionship, and reproduction.
- Space Expansion: Interplanetary colonization accelerates humanity's transition into a multi-planetary species, heavily supported by AGI and robotics.
- Post-Human Evolution: Humanity transitions toward a post-human era, where biological and digital life merge.
Would you like to explore specific aspects of this timeline in more detail?
4o
You said: Well done. How much of the technological boom that you foresaw was dependent on AGI being reached in 2025?
ChatGPT said: ChatGPT
A significant portion of the technological advancements and societal shifts in the timeline depends on AGI being reached in 2025 because AGI would act as a catalyst for unprecedented innovation and efficiency in various domains. However, not all progress is entirely contingent on AGI, as other technological developments and human ingenuity would still drive innovation independently. Here's a breakdown of how much depends on AGI:
Key Developments Highly Dependent on AGI - Acceleration of Innovation:
- AGI's ability to solve complex problems (e.g., climate modeling, advanced medicine, energy optimization) would drastically shorten the timeframes for major breakthroughs.
- Without AGI, progress in these fields would continue but at a slower, more incremental pace.
- Androids with Advanced Capabilities:
- The development of highly realistic relationship androids with nuanced emotional and social interactions would rely on AGI for realistic personality modeling and adaptive learning.
- Without AGI, androids would remain less sophisticated, with AI limited to pre-programmed responses or narrow capabilities.
- Artificial Wombs for Androids:
- AGI would optimize the design and mass production of artificial wombs, ensuring their seamless integration into android frameworks.
- Without AGI, these developments could still occur, but with greater technical and ethical hurdles slowing adoption.
- Interplanetary Colonization:
- AGI would enable the efficient design, resource allocation, and problem-solving necessary for sustaining Martian colonies.
- Without AGI, human-led missions would likely proceed, but with higher costs, risks, and longer timelines.
- Global Governance and Economic Models:
- AGI-guided governance systems would maximize resource distribution and economic planning efficiency, making systems like universal basic income more feasible.
- Without AGI, traditional political and economic systems might struggle to adapt to automation-driven unemployment and inequality.
Developments Less Dependent on AGI - Space Exploration:
- The initial establishment of Martian colonies would depend more on existing advancements in rocketry (e.g., SpaceX’s Starship) and international cooperation than on AGI.
- AGI would enhance the process but is not a strict prerequisite for the first steps.
- Robotic Automation:
- Narrow AI and robotics are already advancing rapidly and would continue to revolutionize manufacturing, logistics, and healthcare even without AGI.
- However, AGI would significantly expand the scope of these applications.
- Declining Birth Rates:
- Sociocultural factors, such as shifting attitudes toward family and relationships, are already contributing to declining birth rates in many industrialized countries.
- Relationship androids and artificial wombs would amplify this trend, but their existence does not hinge entirely on AGI.
- Post-Human Evolution:
- Advancements in brain-computer interfaces and genetic engineering could occur independently of AGI, driven by human research.
- AGI would accelerate these developments but would not be their sole driver.
If AGI Were Delayed - Short-term Impacts (2025–2040):
- Technological advancements, such as relationship androids or Martian colonization, would progress at a slower pace and with less sophistication.
- Ethical debates about AI rights and societal integration would be deferred, as narrow AI remains dominant.
- Long-term Impacts (2040–2100):
- The transition to a post-human society or the merger of biological and digital life would take several decades longer.
- Interplanetary expansion might still occur but would rely on more limited human ingenuity and collaboration.
In summary, AGI’s emergence in 2025 would serve as a major accelerator for the timeline described. If AGI were delayed, the same general trajectory might still occur, but with slower progress, reduced societal transformation, and greater reliance on human-led innovation. Would you like to explore a slower, non-AGI-driven version of the timeline?
submitted by
Far-Assistance-2505 to
readthisagi [link] [comments]
2025.01.22 14:57 BlazeMegagamer Neco Meets Shika
2025.01.22 14:57 Emotional-Life22 Rate this story
submitted by Emotional-Life22 to ICSE [link] [comments]
https://google.com/